Gates' 'value-added' study is full of holes

"Slightly better than a coin toss"

A study released last month by the Gates Foundation supposedly offered up “some of the strongest evidence to date of the validity of ‘value-added’ analysis.” VAA makes the claim that teachers' effectiveness can be reliably estimated by gauging their students' progress on standardized tests.  But Jesse Rothstein, an economist at UC Berkeley, argues that the analyses in the report do not support its conclusions.

Rothstein reviewed the Gates study for the Think Twice think tank review project. His  review is published by the National Education Policy Center, housed at the University of Colorado . Rothstein, who in 2009-10 served as Senior Economist for the Council of Economic Advisers and as Chief Economist at the U.S. Department of Labor, has conducted research on the appropriate uses of student test score data, including the use of student achievement records to assess teacher quality.

He concludes: 

“A teacher who focuses on important, demanding skills and knowledge that are not tested may be misidentified as ineffective, while a fairly weak teacher who narrows [his or] her focus to the state test may be erroneously praised as effective.”

In other words, “teacher evaluations based on observed state test outcomes are only slightly better than coin tosses at identifying teachers whose students perform unusually well or badly on assessments of conceptual understanding."