Skip to content

More Evidence on the Shortcomings of “Value-Added” Teacher Evaluations…

November 7, 2012

Linda Darling-Hammond, whom I wish Barack Obama had chosen as his Education Secretary, is out with an OP-ED in the La Times on how to measure teachers in the classroom.

Measuring teachers is a tricky thing to do as student makeups vary widely from year to year.  What Mrs. Hammond is proposing is that teachers need to be evaluated by other trained teachers, not by the test scores of their students.

I think my readers will really enjoy this article.  Imagine how nice it would have been to have this lady as our nation’s Secretary of Education!

A better way to grade teachers

Effective evaluation requires rigorous, ongoing assessment by experts who review teachers’ instruction, looking at classroom practice and evidence of student learning.

L.A. school district's Academic Growth over Time system

It’s becoming a familiar story: Great teachers get low scores from “value-added” teacher evaluation models. Newspapers across the country have published accounts of extraordinary teachers whose evaluations, based on their students’ state test scores, seem completely out of sync with the reality of their practice. Los Angeles teachers have figured prominently in these reports.

Researchers are not surprised by these stories, because dozens of studies have documented the serious flaws in these ratings, which are increasingly used to evaluate teachers’ effectiveness. The ratings are based on value-added models such as the L.A. school district’s Academic Growth over Time system, which uses complex statistical metrics to try to sort out the effects of student characteristics (such as socioeconomic status) from the effects of teachers on test scores. A study we conducted at Stanford University showed what these teachers are experiencing.

First, we found that value-added models of teacher effectiveness are highly unstable. Teachers’ ratings differ substantially from class to class and from year to year, as well as from one test to the next. For example, teachers who rank at the bottom one year are more likely to rank above average the following year than to rate poorly again. The same kind of wild swings hold true for teachers at the top. If the scores were trustworthy measures of a teacher’s ability, this would not occur.

Second, teachers’ value-added ratings are significantly affected by differences in the students who are assigned to them. Even when statistical models try to control for student-demographic variables, teachers are advantaged or disadvantaged based on the students they teach. Contrary to proponents’ claims, these models reward or penalize teachers according to where they teach and what students they teach, not just how well they teach.

We found that a teacher receives a higher value-added score when he is teaching students who are already higher-achieving, more affluent and more versed in English than when he is assigned large numbers of new English learners and students with fewer educational advantages. In fact, when we looked at high school teachers who teach different classes, the student composition of the class was a much stronger predictor of the teacher’s value-added score than the teacher. This makes sense: With a classroom full of more-advantaged students, teachers can move faster and cover more material, something the statistical models used for value-added ratings fail to capture.

The above two paragraphs really say it ALL!  When you are working with economically disadvantaged students “value-added” measures’ statistical machinations can’t make up for the fact that many of these children have challenges that they face at home and thus, many are focused on survival…not reading…

Finally, value-added ratings cannot disentangle the many home, school and student factors that influence learning gains. These matter more than the individual teacher in explaining changes in scores.

These findings have been replicated in many studies. As a result, most researchers have concluded that value-added scores should not be used in high-stakes evaluations of individual teachers. As the country’s leading research organization, the National Research Council, concluded: “VAM estimates of teacher effectiveness … should not be used to make operational decisions because such estimates are far too unstable to be considered fair or reliable.”

What is the alternative? Certainly we need teacher evaluation systems that identify both excellent and struggling teachers based on what they do and how their students learn. And we need systems that help teachers improve, target assistance where needed and remove teachers who cannot, with help, succeed in the classroom.

California’s Educator Excellence Task Force recently released a report that outlines the most successful practices internationally. It illustrates that, as in other professions, good evaluation starts with rigorous, ongoing assessment by experts who review teachers’ instruction based on professional standards. Evaluators look at classroom practice, plus evidence of student learning from a range of classroom work that includes (but is not limited to) school or district tests that directly connect with the curriculum and students.

Studies show that feedback from this kind of evaluation improves student achievement because it helps teachers get better at what they do. Systems that sponsor the effective Peer Assistance and Review program also identify poor teachers, provide them intensive help and remove them if they don’t improve.

If we really want to improve teaching, we should look to develop such models of effective evaluation rather than pursuing problematic schemes that mis-measure teachers, create disincentives for teaching high-need students, offer no useful feedback on how to improve teaching practice and risk driving some of the best educators out of the profession.

Linda Darling-Hammond, a professor of education at Stanford University and co-faculty director of the Stanford Center for Opportunity Policy in Education (SCOPE), co-chaired the California Educator Excellence Task Force. Edward Haertel is a professor of education at Stanford University and chairman of the National Research Council’s Board on Testing and Assessment.

Advertisements

From → Archives

%d bloggers like this: