The education research of recent years has pointed overwhelmingly to the importance of teachers. Perhaps more than anything else – quality of principal, size of school, size of class – the strength or weakness of classroom teachers influences how much students learn and even how they fare later in life.
The great unknown is how to improve teacher quality, be it by attracting more good teachers, weeding out more bad teachers or helping teachers become better at their craft.
A new study, released on Thursday, offers powerful if still tentative evidence that teacher-evaluation programs can play an important role. The study is especially notable because past research about evaluation programs suggested they had little effect. The new paper, however, studies an evaluation program – called Impact, in the District of Columbia school system – that is far larger, with bigger rewards and stiffer penalties, than most programs.
Impact, which began under Michelle Rhee while she was chancellor, has been a hotly debated program, and the new study is sure to attract attention from both supporters and critics of teacher evaluation. New York state’s plan to begin evaluating teachers has also been the subject of intense praise and criticism, as have such programs elsewhere.
The study found that Impact caused more low-performing teachers to leave the school system than otherwise would have been expected. The program also seemed to improve teaching quality – as measured by classroom observations and test scores – of teachers with both strong and weak evaluation scores.
“High-powered incentives linked to multiple indicators of teacher performance can substantially improve the measured performance of the teaching work force,” conclude the researchers, Thomas Dee of Stanford University’s Graduate School of Education and James Wyckoff of the Curry School of Education at the University of Virginia. Evaluation programs, they add, can bring “substantive and long-term educational and economic benefits” both by “avoiding the career-long retention of the lowest-performing teachers and through broad increases in teacher performance.”
The study has not yet undergone peer review. It is being published as a working paper by the National Bureau of Economic Research, a Cambridge, Mass., group run by some of the country’s top academic economists.
The Impact program differs from some other teacher-evaluation programs in significant ways. While others offered bonuses of several thousand dollars for high performance – up to $3,000 in New York City and up to $6,000 in a Texas district, for example – Impact’s bonuses reached as high as $25,000. Teachers who were judged as “highly effective,” which translated into being in roughly the top 16 percent, in two consecutive years did even better: they received raises that lifted their base pay by almost 30 percent over the next 15 years.
Impact also applied to most of the District of Columbia’s school system, meaning it received far more attention than the smaller pilot programs put in place elsewhere. And Impact’s evaluations were based mostly on classroom observations that measured how well teachers followed so-called best practices. Test scores played only a secondary role, along with principals’ assessments of the teachers’ contributions to the entire school. Mr. Dee and Mr. Wyckoff speculate that using classroom observations may have a larger effect on teachers, because they can more easily control their own teaching style than they can influence their students’ test scores.
To study the program’s effect, the researchers compared teachers whose evaluation scores were very close to the threshold for being considered a high performer or a low performer. This general method is common in social science. It assumes that little actual difference exists between a teacher at, say, the 16th percentile and the 15.9th percentile, even if they fall on either side of the threshold. Holding all else equal, the researchers can then assume that differences between teachers on either side of the threshold stem from the threshold itself.
The results suggest that the program had perhaps its largest effect on the rate at which low-performing teachers left the school system. About 20 percent of teachers just above the threshold for low performance left the school system at the end of a year; the odds that a teacher just below the threshold would quit were instead above 30 percent.
In addition, low-performing teachers who remained lifted their performance, according to the system’s criteria. To give a sense of scale, the researchers noted that the effect was about half as large as the substantial gains that teachers typically make in their first years of teaching combined.
The program’s effects were not as widespread on teachers judged to be high performing. They were only slightly more likely to remain in their jobs than similar teachers who just missed the cutoff, and the effect was not large enough to be statistically significant. But high performers did lift their performance significantly after being deemed a top performer.
Teachers who were rated “highly effective” one year lifted their performance to an even higher level, presumably to win the permanent increase in pay that came with being deemed “highly effective” two straight years. These teachers, Mr. Dee told me, showed “clear gains in the quality of their classroom practice and in how their principal rates their contributions to the school community.” The progress their students made on test also rose somewhat, he said.
The researchers were careful to note that their study is not the last word on teacher-evaluation programs. For one thing, the study focused on teachers near the cutoff for high and low performers; as a result, it says less about the effect of evaluation programs on the very best and the very worst teachers, as well as those in the middle.
But the study is still significant enough to change thinking about evaluation programs. Barring new evidence that suggests otherwise, it indicates that a broad-based evaluation program, with large performance incentives, will indeed affect teachers – and, more important, students.
And that finding should probably not be surprising. Teachers are not unique. They have a lot in common with doctors, athletes, auditors, real-estate agents and people in many other lines of work. They’re human: when their work is evaluated and they are held accountable for it, they tend to do a better job.
Source: http://economix.blogs.nytimes.com/2013/10/17/a-new-look-at-teacher-evaluations-and-learning/?pagewanted=print
