My 11.125 students read education articles each week and discuss current events for about 15 minutes. It's a lot of time to commit to the endeavor, but I think it's a great way, in a survey class, to let students explore their interests and to learn more about the various issues and challenges in education.
This week we read a blog post, "The Test Score Hypothesis" commenting on the Chetty et al study concerning value added scores, test scores, and life outcomes such as income and teenage pregnancy. The thrust of the blog post is that Chetty's study is one of the most important studies to draw a link between standardized test scores and life outcomes-- that is, teachers who are good at boosting test scores are also good at boosting life outcomes. It is a powerful (perhaps not conclusive) piece of evidence that we can identify teachers who help students have a better life by identifying teachers who help students score better on tests.
One of my very students in class asked two questions which I thought were quite good skeptical questions, and I thought I'd share my responses.
First, she asked "Aren't they potentially confusing correlation with causation here?"
The answer to that is: "No. They have a method which they claim allows them to make causal assertions. Basically, they assert that the particular years in which teachers enter and leave a position are randomly determined, and therefore, you can leverage teacher mobility to make causal claims." Now, you might come up with all kinds of reasons to assert that this isn't actually random (if you have a bone to pick with the reform crowd, I recommend pointing out that they made a big deal in Waiting For Superman about how teacher mobility is, in part, a function of poor performance, the dance of the lemons or whatever it was called). Personally, I do think there's some good random in there. But they are not confused. Wrong maybe, but not confused.
Second, she asked "How much effect do teachers have anyways? I mean, don't kids from successful parents go to successful schools and so on?" This is an astute observation, and indeed, socioeconomic status remains a powerful predictor of student performance. But teachers can have a powerful impact on any group of students, and Chetty's models do account for this by controlling for school effects and controlling for SES. So even though wealthy and poor kids start at different places, on average, every year, we can still see how much teachers can push kids towards further achievement.
I think there are a methodological critiques you can hurl at Chetty and his co-authors, but on the whole, I'm pretty impressed both with how thoroughly they engage challenges brought up in the prior literature and how forthright they are in places where there are limitations in their approach.
To me, the big looming issue over Chetty is Campbell's Law-- that measures become less useful when society incentivizes those measures. In other words, Chetty's study takes place in a low-stakes world, will it still hold up in a high-stakes testing culture? Will test scores still be a good way to identify good teachers when we have all of them focus, increasingly exclusively, on test scores.