Goodhart's Law of unintended consequences

Goodhart's law is named after the banker who originated it, Charles Goodhart. Its most popular formulation is: "When a measure becomes a target, it ceases to be a good measure."

In other words: “When you put a lot of weight on one measure, people will try to do well on that measure,” says Jonah Rockoff of Columbia. “Some things they do will be good, in line with the objectives. Others will amount to cheating or gaming the system.”

What are the consequences of grading teachers by the test? Economics writer Eduardo Porter's NYT piece refers to Goodhart's Law in the context of using high-stakes, standardized tests to grade teachers.

Luis Garicano at the London School of Economics calls it the Heisenberg Principle of incentive design, after the defining uncertainty of quantum physics: A performance metric is only useful as a performance metric as long as it isn’t used as a performance metric.

Porter quotes Randi Weingarten:

“People who claim to be market-based reformers want to sell the theory that there is a direct correlation between test scores, the effort of teachers and the success of children,” said Randi Weingarten, who heads the American Federation of Teachers. “It just ignores everything else that goes into learning.”

I only wish that Randi would have remembered that when she signed on to Arne Duncan's call for more and earlier national high-stakes testing. AFT’s position is detailed in a joint statement issued with the Center for American Progress, a Democratic Party aligned think tank.

“We propose to keep annual tests so parents have valid information about their children’s progress but want to ensure that any school accountability a system has a broader array of indicators that fully captures how our children are learning,” said CAP President Neera Tanden.

Randi and CAP's assumption here is that tests like the PARCC can really provide "valid information" even with Goodhart's Law in play.

Porter says:

Teachers argue there is no way they could isolate the impact of teaching itself from other factors affecting children’s learning, particularly such things as the family background of the students, the impact of poverty, racial segregation, even class size.   

As usual, the teachers are right.

Porter wimps out at the end of his piece, quoting rabid testing proponent and former NYC Chancellor Joel Klein who calls for reliance on so-called "value-added scores" in order "to penalize or reward teachers". Porter, like Weingarten, calls for grading teachers on student test scores along with "other measures." That is the current approach in most districts these days.

But even with the mixed-measures approach, the high-stakes tests when used to determine teachers' and administrators' salaries have more power than any of the softer measures of student/teacher performance and therefore are really all that counts. The unintended consequences include gaming the system and teaching to the tests. It also ignores the role that poverty and other out-of-school factors play on measurable student/teacher performance.

High-stakes, standardized testing is part of the problem. Not part of the solution. It shouldn't be used, even in combination with other more valid measures, to grade teachers.