Taking the fall for Duncan's testing madness

SEE A PATTERN HERE?...On their eighth day of deliberations, the jurors convicted 11 of the 12 Atlanta black educators and staff of racketeering, a felony that carries up to 20 years in prison. 

One thing you can count on is that when student test scores become education's stock-and-trade, there will be more cheating scandals like the one it Atlanta. In fact, they've already reached pandemic proportions. You can also count on educators being the ones to take the fall and even go to prison when the system itself is set up to reward gaming.

Watch this powerful scene from The Wire to better understand how the whole system is built on official and unofficial cheating.



As I pointed out last week, the real law in operation here is Goodhart's Law of unintended consequences. It goes something like this: "When a measure becomes a target, it ceases to be a good measure."

Teachers are even being evaluated on high-stakes test scores from kids and in subjects they don't even teach, writes Valerie Strauss at WaPo:

For example, an art teacher in New York City explained in this post how he was evaluated on math standardized test scores, and saw his evaluation rating drop from “effective” to “developing.” High-stakes tests are only given in math and English language arts, so reformers have decided that all teachers (and sometimes principals) in a school should be evaluated by reading and math scores.

Sorry Arne Duncan, but using so-called "value-added models" (VAM) doesn't make things any better. So says the American Statistical Association .

Estimates from VAMs should always be accompanied by measures of precision and a discussion of the assumptions and possible limitations of the model. These limitations are particularly relevant if VAMs are used for high-stakes purposes.  — John Merrow (@John_Merrow) April 2, 2015

WEEKEND QUOTABLES

CTU Pres. Karen Lewis asks NPE conferees, "If you support the [test-boycotting] teachers at Saucedo & Drummond, stand up and say aye."

Diane Ravitch 

"A for-profit public school is an oxymoron." -- Speech at NPE Conference

Mark Woods

I’m not even sure where to begin. But let’s start with the music, P.E. and art teachers. Each had a negative VAM score, based largely on student tests in subjects that they don’t even teach. Beyond that obvious flaw, there is seemingly overlooked value of simply having elements of music, P.E. and art... Florida x VAM = SHAM. -- Florida Times Union

Ben Joravsky

 Wait—another update! Teachers at Drummond Elementary—located on the north side—just voted not to give the ISAT. Despite B3's ultimatum. Oh, no. The rebellion's spreading. What's an all-powerful mayor to do?... Here's what you should do, Mr. Mayor. Stick to Polar Plunges with Jimmy Fallon and leave the schools alone. Haven't you learned? Every time you intervene, you only make things worse. -- Chicago Reader

Dawn Neely-Randall, a 24-year veteran teacher in Ohio

But I have to tell you, school just isn’t for children anymore. -- Washington Post

The war on teachers: Test 'em all, let God sort 'em out

Arne Duncan demands Value-Added teacher evaluation asking, “What’s there to hide?”

BUSH MENTALITY...Remember when Bush and Cheney decided to invade Iraq even though they knew Iraq had nothing to do with the 9/11 attacks and that there were no WMDs? Now that same mentality is being used by the current administration, only in this case, it's applied to teacher evaluation. A stretch, you say? Take a look. 

Well we already know how crazy it is to rate teachers' performance on the basis of their students' standardized test scores, using this insane Value-Added metric.

y = Xβ + Zv + ε where β is a p-by-1 vector of fixed effects; X is an n-by-p matrix; v is a q-by-1 vector of random effects; Z is an n-by-q matrix; E(v) = 0, Var(v) = G; E(ε) = 0, Var(ε) = R; Cov(v,ε) = 0. V = Var(y) = Var(y - Xβ) = Var(Zv + ε) = ZGZ^T + R.

But here's where that kill-'em-all-let-God-sort-'em out mentality comes in to play. In Florida, where Bush Brother Jeb ruled the roost for years, about 70% of the Florida teachers are being evaluated, and often merit-paid, promoted or fired using scores based on test results from students they never taught and/or in subjects they don’t teach. 

According to Washington Post columnist Valerie Strauss: 

There are numerous problems with using VAM scores for high-stakes decisions, but in this particular release of data, the most obvious and perhaps the most egregious one is this: Some 70 percent of the Florida teachers received VAM scores based on test results from students they didn’t teach and/or in subjects they don’t teach. 

Yes, you read that right: Teachers are being evaluated on students they didn’t teach and/or subjects they don’t teach. How can that be?

Yes, how can that be? For an answer, one might turn to former Defense Sec. Rumsfeld, who after 9/11 gave the order, "Go massive . . . Sweep it all up. Things related and not.”

Different administration, I know. Same mentality, only now there's a war directed at teachers.

Evaluate teachers -- things related and not. 

Gates threatens to pull $40M in Pittsburgh funds unless union caves on VAM

I'm not sure if this has anything to do with AFT Pres. Randi Weingarten's sudden turn against so-called Value-Added Models (VAM) of teacher evaluation or not.

But I see that in Pittsburgh, Bill Gates is threatening to renege on $40 million he promised the district if the teachers union and PPS don't "work in harmony" to develop a VAM plan in which teachers are evaluated and paid on the basis of their student's test scores. Collaboration in this case obviously means that the city's teachers are forced to accept anything the district throws at them for fear of being blamed for losing the Gates money.

Gates pulled the same crap in D.C.

You may remember that Gates, Broad, and Walton pulled the same crap in D.C. when they, along with Arne Duncan,  threatened to pull out millions in district funding if voters booted out Mayor Fenty and Michelle Rhee. They did.

But Fenty's replacement Vincent Gray and his current Supt. Kaya Henderson (Rhee's assistant) have bowed to the power philanthropists and continued Rhee-ism (including VAM), without Rhee.

This is precisely why AFT rank-and-filers were concerned with when Weingarten accepted millions from Gates for the union's Innovation Fund. "Innovation" has become practically synonymous with VAM.

This is what happens when public schools are forced to go begging to the world's richest man in order to keep "public" education alive. The question now is, are Weingarten and the PFT willing and able to stand up to Gates' blackmail.

I had an interesting give-and-take with Randi yesterday after she Tweeted:

@NEPCtweet @mikeklonsky-I have always been leery abt #VAM-it's in contracts b/c RTTT-compelled state law- not wking in places its been tried
— Randi Weingarten (@rweingarten) 

I responded:

@rweingarten If you were "always leery" why did you praise Cleveland #VAM deal allowing dist to fire teachers after 2 yrs of bad evals?
— Mike Klonsky (@mikeklonsky) 

And:

@rweingarten @NEPCtweet But you called New Haven contract "a model" "a template". Doesn't sound like you were "leery" abt #VAM. Does it?
— Mike Klonsky (@mikeklonsky) 

And it went on from there. More probing her always leery comment on my part. No real responsiveness on her part, especially about the damage already done in New Haven, Cleveland, N.Y. and now in Pittsburgh. So much for the power of the Tweet.

Randi's "VAM is a sham" epiphany

Weingarten, Bloomberg, and Klein announce an agreement on plan that would give N.Y. teachers bonuses based on the test scores of students at schools that have high concentrations of poor children. (New York Times)

I'm glad AFT Pres. Randi Weingarten has finally seen the light on VAM. Really I am.

For those unfamiliar with so-called Value Added Modeling, it's the statistical measure that judges teacher quality based on the test scores of their students. The formula for evaluating teachers, deciding their performance-based pay, or even whether they fired from their jobs, looks like this.

y = Xβ + Zv + ε where β is a p-by-1 vector of fixed effects; X is an n-by-p matrix; v is a q-by-1 vector of random effects; Z is an n-by-q matrix; E(v) = 0, Var(v) = G; E(ε) = 0, Var(ε) = R; Cov(v,ε) = 0. V = Var(y) = Var(y - Xβ) = Var(Zv + ε) = ZGZT + R.

It was obviously invented by some mad scientists and mathematicians working in the basement of the Gates Foundation. 

But the question I have for Randi is -- what are you going to do about all those VAM-based contracts you went along with, lobbied for and helped impose on thousands of teachers in districts like New Haven (which you hailed as a "model" and a "template"),  New York,  L.A., Cleveland, Detroit, and D.C.?

You may have changed you mind, but we will be living with VAM for years to come.

Still cleaning up Rhee's Value-Added mess in D.C.

Seattle teachers protest Rhee-forms.

They're still cleaning up the debris from Michelle Rhee's signature reform in D.C.. Her top-down imposed Value-Added IMPACT system of teacher evaluation has long caused havoc in the system here and elsewhere. Remember, it's also a central piece of Arne Duncan's Race To The Top.

The teachers union should have never agreed to IMPACT and evals based on student test scores.  The formula for tying every student's test score bump or dip directly to an individual teacher is so complicated and inherently error-ridden that it's bound to confuse parents and negatively affect teachers and students.

Monday's Washington Post reports that faulty calculations of the “value” that D.C. teachers added to student achievement in the last school year resulted in erroneous performance evaluations for 44 teachers, including one who was fired because of a low rating. School officials described the errors as the most significant since the system launched a controversial initiative in 2009 to evaluate teachers in part on student test scores.

Half of the evaluations for the 44 teachers were too high and half too low, said Jason Kamras, chief of human capital for D.C. Public Schools.

Current WTU Pres. Elizabeth Davis, called the disclosure disturbing.

“IMPACT needs to be reevaluated,” Davis said. “The idea of attaching test scores to a teacher’s evaluation — that idea needs to be junked.”

Now she tells us.

D.C. Mayor Vincent Gray and Chancellor Kaya Henderson, a Rhee protege, aren't likely to junk V-A regardless of its consequences.

We should also consider the shameful role played by some academics in the imposition of value-added evals. One case in point: It was back in October when Professors Thomas Dee of the Stanford Graduate School of Education and James Wyckoff of the University of Virginia were heaping praise on  Rhee's V-A program, calling it, effective and accurate.

LONG WEEKEND QUOTABLES

Joe Biden at the NEA RA

"There is an organized effort to place blame for budget shortfalls on educators and other public workers. It is one of the biggest scams in modern American history." -- Education votes

Teacher Ken

“Teachers are being targeted,” said Ken Bernstein, a Prince George’s County teacher who is helping to coordinate the march. “And they are finally coming out of their classrooms and getting interested” in organizing a political response.  -- Washington Post

Brother Fred

Fred Klonsky, a Chicago-area teacher and delegate to the convention, said the Obama administration should take note of the 72% vote. “For an incumbent Democratic president to receive less than 75% of the delegate vote at an NEA convention ought to cause some concern for the administration,” he said. -- WSJ

Joanne Barkan

"With the zealots’ mix of certainty and fervor, ed reformers have made this a wretched time to be a public school teacher." -- "The Grand Coalition Against Teachers"

 More seat time?

"Just extending a school day doesn’t mean, by itself, that you’re going to have high-performing schools. It’s not going to be simple.”  -- Noemi Donoso, new CPS chief education officer

The 'value-added' debacle

Leading mathematician debunks use of  VAM 

The L.A. Times is once again publishing the names of every teacher in the district next to a concocted "value-added score." The reputations and careers of thousands of teachers hang in the balance with the Times' use of a bogus formula they claim can distinguish good teachers from bad ones, who should get "merit pay"and who should lose their jobs.

In a scene out of the theater of the absurd, even Supt. John Deasy and other district leaders are protesting the Times' release of this data, claiming the district is using a different, competing formula from the one used by the Times. Here's Deasy' formula:

y = Xβ + Zv + ε where β is a p-by-1 vector of fixed effects; X is an n-by-p matrix; v is a q-by-1 vector of random effects; Z is an n-by-q matrix; E(v) = 0, Var(v) = G; E(ε) = 0, Var(ε) = R; Cov(v,ε) = 0. V = Var(y) = Var(y - Xβ) = Var(Zv + ε) = ZGZ^T + R.

The Deasy letter, obtained by education gadfly Alexander Russo and posted on his blog This Week in Education, warned:  “This is very likely to create confusion for many educators and parents.” [See the whole letter here.]

Here's what mathematician, John Ewing, president of Math for America has to say about the Times' bad math:

When value-added models were first conceived, even their most ardent supporters cautioned about their use [Sanders 1995, abstract]. They were a new tool that allowed us to make sense of mountains of data, using mathematics in the same way it was used to understand the growth of crops or the effects of a drug. But that tool was based on a statistical model, and inferences about individual teachers might not be valid, either because of faulty assumptions or because of normal (and expected) variation. -- WaPo Answer Sheet 

Despite this apparent lack of validity, the use of "value-added" formulas as the basis for evaluating, paying and firing teachers, is rapidly becoming the new face of corporate reform with full approval coming from Arne Duncan and the DOE.

How Deasy will decide who's a good teacher

Here's his value-added formula

Want to know how L.A.'s incoming supt. John Deasy (the man from Gates) is going to sort good teachers from bad? How he's going to decide who gets fired and who stays? Who has their careers destroyed? Who will get paid? How much?

It's all right here:

y = Xβ + Zv + ε where β is a p-by-1 vector of fixed effects; X is an n-by-p matrix; v is a q-by-1 vector of random effects; Z is an n-by-q matrix; E(v) = 0, Var(v) = G; E(ε) = 0, Var(ε) = R; Cov(v,ε) = 0. V = Var(y) = Var(y - Xβ) = Var(Zv + ε) = ZGZ^T + R.

Gates' 'value-added' study is full of holes

"Slightly better than a coin toss"

A study released last month by the Gates Foundation supposedly offered up “some of the strongest evidence to date of the validity of ‘value-added’ analysis.” VAA makes the claim that teachers' effectiveness can be reliably estimated by gauging their students' progress on standardized tests.  But Jesse Rothstein, an economist at UC Berkeley, argues that the analyses in the report do not support its conclusions.

Rothstein reviewed the Gates study for the Think Twice think tank review project. His  review is published by the National Education Policy Center, housed at the University of Colorado . Rothstein, who in 2009-10 served as Senior Economist for the Council of Economic Advisers and as Chief Economist at the U.S. Department of Labor, has conducted research on the appropriate uses of student test score data, including the use of student achievement records to assess teacher quality.

He concludes: 

“A teacher who focuses on important, demanding skills and knowledge that are not tested may be misidentified as ineffective, while a fairly weak teacher who narrows [his or] her focus to the state test may be erroneously praised as effective.”

In other words, “teacher evaluations based on observed state test outcomes are only slightly better than coin tosses at identifying teachers whose students perform unusually well or badly on assessments of conceptual understanding."

Chicago forum on teacher evaluation

I even put a tie on

It's the day after Halloween so I didn't feel so bad about putting on a tie and dressing up like a downtown school reformer. The reason? It's the first of this year's Schools Policy Luncheon Series, organized jointly by BPI and Catalyst. This one, titled: Teacher evaluation and compensation: Getting it right, grabbed my interest for obvious reasons. The biggest crowd ever, packed the main lounge at the Union League Club (business attire required). Thus the tie.

Lunching on sliced chicken, salad greens, chocolate cake with a blackberry next to it, and lots of coffee, I heard Susan Moore Johnson from Harvard, CTU prez, Karen Lewis, and Peter Martinez, the head of UIC's Urban Leadership Program.

Johnson, coming from Harvard, of course had a Power-Point nobody could read. But her assessment of Value-Added based on standardized tests, "which measure only part of what teachers are expected to do," was great. She basically argued that VA and standardized tests should NOT be used to make important decisions about schools firing or paying teachers; that VA doesn't show teachers how to improve; and that it doesn't improve collaboration within schools. She hit hard on the L.A. Times publication of teacher names next to student scores. What we need, said Johnson, is more peer evaluation and review with support from skilled experienced master-teachers, set up with oversight by a labor-management review panel. Wow! Not bad.

She was followed by Lewis, who spoke with passion and conviction about the role of the teachers union in leading school improvement. Her main points followed seamlessly from Johnson's research and she concluded, "you can't test or fire your way to good schools." This got some applause from about 20% of the room and some eye-rolling from about another 20%. But she seemed to have consensus when she pointed out that the system of teacher evaluation as we know it, was broken.

Fix it, yes, she said.. But even that won't do without a wrap-around system of social services to improve the conditions kids and families face outside of school.

Martinez seemed to be the least critical of standardized testing--"that's what people understand." But he did call for a more collaborative approach based on high standards and the need for good instructional strategies. Okay! No arguments from anyone there, I suppose.

So I left the meeting full of buzz from all the caffeine and chocolate in my system, wondering why, if all these movers and shakers, union leaders and business reformers could reach basic agreement around the need for improved teacher evaluation, and a critique of value-added and standardized testing, why are we still reinforcing merit-pay based on a faulty standardized testing regimen here in Chicago?

Why are the union and a handful of us activist types the only ones speaking out in opposition? Maybe it's too dangerous given Race To The Top and the crack-down on school districts that step out of line.

Collabaorative Planning

He forgot to say, "Value-added"

Thanks to Larry Ferlazzo and brother Fred for forwarding this video.

Fighting the pissants

Leave it to Alan Gottlieb from the Education News Colorado, to accuse my teacher/blogger brother Fred of trotting canards. We in the Klonsky family don't like to talk about this publicly, but since the L.A. Times set the tone by publishing the names and test-score rankings of individual teachers, I think it's only right, in the name of transparency, to inform parents that Fred has had this trotting-canards problem since his was a kid. My folks took him to the best doctors, but, alas, no cure was found. If some parents decide to seek another teacher for their child, it's perfectly understandable.

Fred quacks back: “Trotting out the canard?” Isn’t “canard” French for goose?

Posting at Huff, Gottlieb has even more brilliant observations to offer us re. the L.A. Times value-added fiasco, starting with his classifying all public school teachers as "insiders" and "choice" proponents, corporate reformers and Duncan as "outsiders" He goes on to admit that, "the methodology may be imperfect. Some teachers can't be evaluated based on value-added criteria. Yes, some embarrassment will result," but what the hell, publish anyway.

They used to teach ethics back when I was in journalism school. I guess things have gotten more technical and innovative since then, especially out there in Colorado, where they've dropped trow as low as it goes in hopes of attracting some Race To The Top money.  So far, no good.

Arne Duncan takes, pretty much, the same line as Gottlieb (or vice versa). He's critical of the poor tests we use. He doesn't think that these tests should be the only criterion for evaluating teachers. Yada, yada, yada. But still, he applauds the L.A. Times for doing just that--allthough with some qualifications. As the New York Times reports: 

In a speech last week, though, he qualified that support, noting that he had never released to news media similar information on teachers when he was the Chicago schools superintendent.“There are real issues and competing priorities and values that we must work through together — balancing transparency, privacy, fairness and respect for teachers,” Mr. Duncan said. On The Los Angeles Times’s publication of the teacher data, he added, “I don’t advocate that approach for other districts.” 

Another soldier in the pissant army is Jack Shafer. Writing in Slate, Shafer, salutes the Times for "bravery  to express in liberal, union-enslaved Los Angeles."

What can you say in response to that? Except, thank god we live in a country where even pissants can publish. 

Even Hess had a problem with L.A. Times story

The right-wing think-tank's man at Edweek, Rick Hess, liked the L.A. Times' singular use of student standardized-test scores and value-added assessment, to rank teachers. But even Hess was somewhat critical.

I'm all for building and refining these systems and using them to evaluate, reward, and remove teachers. But I think it's a mistake to get in the business of publicly identifying individual teachers in this fashion.

Hess' criticisms makes me wonder, just how far to the right does one have to be to really get behind this latest Times debasement of city teachers?


Value added?

There was one part of Hess' post that gave me a chuckle. It was his comment on public transparency:  

"It typically doesn't entail reporting on how many traffic citations individual LAPD officers issued."

I'm laughing only because I'm reading the front-page story in the Sun-Times about a memo from Mayor Daley's office (the man with singular power over our public schools) ordering Chicago cops to write more tickets. It's accompanied by a sidebar story: "Commanders told to make lists of worst cops" which reads as follows:

"Over the last year, officers have undergone evaluations on everything from the number of tickets they write to the number of arrests they make. But they don't know the standards for winding up on a list of worst performers."

You should understand that in the past year, Daley, in full privatization mode of any bit of public space that isn't nailed down, sold the city's parking meters to a private firm. The resulting anger from Chicago masses have left Daley popularity ratings down in the dirt.

It sounds now like City Hall has taken Hess' counsel to heart -- it's a kind of L.A. Times value-added approach without the actual naming of names in the press. It shows that VAA works sometimes, depending of course on what it is you value. In this case writing more tickets and thereby generating for cash for the city's coffers. Or in the case of the schools, a singular focus on test scores as a new gold standard in teacher evaluation.

******

Aftermath

Before I could finish writing this post, Daley went ballistic in response to criticism from the S-T story. The mayor now claims he knew nothing about the City Hall ticket quota memo.

"Stupidity. Just stupid. Just stupid. Some bureaucrat sent that out. That's all it is." (WBEZ)

And guess what? Daley is calling for the name of the poor schmuck who wrote the memo. No value-added, .

Duncan in hog heaven over L.A. Times story

He liked the mass firings of teachers at Central Falls. But that was nothing compared to this  L.A. Times story, posing as research. He loves it.

Spurred by the administration, school districts around the country have moved to adopt "value added" measures, a statistical approach that relies on standardized test scores to measure student learning. Critics, including many teachers unions and some policy experts, say the method is based on flawed tests that don't measure the more intangible benefits of good teaching and lead to a narrow curriculum. In Los Angeles, the teachers union has called public disclosure of the results "dangerous" and "irresponsible." 

Ah, perfect, says Duncan.

Bracey's debate with William Sanders

In light of the current response/debate over the L.A. Times misuse of Value-Added Assessment of teachers, I am posting this 2007 Huffington column by the late Gerald Bracey, in full.--M.K.

Value Subtracted: A "Debate" with William Sanders

 I knew in advance that things would be weird at this debate, symposium, whatever with William Sanders. About three years ago in my Phi Delta Kappan research column, I summarized an article critical of Sanders' value-added assessment model. Sanders visited upon me a two-hour telephone explanation-harangue of why I was wrong. He visited upon the editors a lonnnggg letter. On learning that I would be the other principal speaker, Sanders insisted that he have the last word. Otherwise, he wouldn't come. Can you accept these terms, my host at North Carolina State wanted to know. I said I could.

Sanders made a presentation which had virtually nothing to do with anything that he is known for, which is using value-added assessment (VAA) to determine which teachers are effective. I had been less than enamored with this from the beginning since his first model used off-the-shelf items from McGraw-Hill's CTBS. Wait, you're using norm-referenced test items to pass judgments on teachers? Oh, please. In this talk, though, he did not consider the background knowledge of the listeners, most of whom were teachers hearing about value-added for the first time and one could almost see the bullets of jargon zipping past their ears.

A value-added model tests students at the beginning of the year and at the end. The change in test scores over the year is the "value" that has been added. The question then becomes: how much of this added value does the teacher account for (as opposed to what is added by parents, community, etc.)?

My points were these:

VAA makes more sense than the current successive-cohorts system for determining AYP. It makes more sense to follow kids over time, although if the goal remains 100% proficiency the whole operation remains nuts.

VAA is circular: it defines effective teachers as those who raise test scores, then uses test score gains to determine who's an effective teacher.

Aside from Sanders, those working in VAA (Henry Braun, Howard Wainer, Dan McCaffrey, Dale Ballou, J. R. Lockwood, Haggai Kupermintz, from all of whom I had quotes) acknowledge that it cannot permit causal inferences about individual teachers. At best, it is a beginning step to identify teachers who might need additional professional development.

It is regressive in that it reinforces the idea that schools have teachers in boxes with 25 kids. Sanders claims his technique can deal with team-taught classes, but even if that is true, and he offered no data, it misses the dynamic of schools. As Kupermintz put it, "The TVAAS model represents teacher effects as independent, additive and linear. Educational communities that value collaborations, team teaching, interdisciplinary curricula and promote student autonomy and active participation may find [it of little use]. It regards teachers as independent actors and students as passive recipients of teacher 'effects'..." In fact, as class size gets smaller, the TVAAS makes it harder for a teacher to look outstanding or ineffectual.

Sanders' model improperly assumes that educational tests form equal-interval scales, but they do not and no amount of finagling with item response theory will fix that. On a thermometer, a true equal interval scale, the amount of heat needed to go from 10 degrees to 11 is the same as that needed to go from 110 to 111. On a test, it might require very different amounts of "achievement" to get from one point to another on different parts of the scale. Sanders believes that using NCE's cures this (ha). It presumes that the teacher "effect" persists -- like a diamond, it lasts undiminished forever. I'd like to run that by a few cognitive psychologists. It presumes that academic achievement is unidimensional.

And, perhaps most crucially, it presumes that students and teachers are randomly assigned to classes and overlooks that they are not. Many people choose a school by choosing where to live and within districts they sometimes choose a school other than the neighborhood school. Teachers with seniority get to choose what school or what classes they teach. They don't usually choose hard-to-teach kids. And parents exert pressure--here, parents kill to get their kids into Pat Welsh's high school writing classes. Big changes in test scores might well reflect these deviations from randomness as much as anything teachers do in their classrooms. Value-added models typically act as if this isn't important. It is.

Worst, even ignoring its failures, value-added might not give stable results. An article by J. R. Lockwood and others in the Spring, 2007 issue of the Journal of Educational Measurement finds that, using a test that tests mathematical procedures, they could generate a list of effective teachers. Using a test of math problem solving they could generate a list of effective teachers. But they weren't the same lists!

Value-added is currently being oversold. At the Battelle for Children website, one read, "Combining value-added analysis and improved high school assessments will lead to improved high school graduation rates, increased rigor in academic content, high college going rates less college remediation and increased teacher accountability." And how many validity studies support these assertions?

Sanders' 15 minutes of last word was a rambling, illogical lecture of the type a father might visit on a prodigal. The sponsors were embarrassed, the audience was pissed. At the reception that followed, for a while Sanders sorta took over a group I was talking with and I concluded that Sanders has an extremely limited yet extremely rigid idea of how schools work (his doctorate is in biostatistics and he worked with the Atomic Energy Commission and in agriculture until the late 80's), rejects any conclusion counter to his own and, in spite of his age, somewhere around 75, is as defensive as any novice.

Posted: May 1, 2007 06:20 PM 

THE VALUES BEHIND VALUE-ADDED

The LA Times in cahoots with Rand researchers, and the Hechinger Inst. ran an article Sunday that's the first in a series leading to the release of scores of teacher effectiveness based on a value-added model. If the point, as the Times claimed, was to show the importance of having top-notch teachers in classrooms, especially those classrooms with the neediest kids, I would have been okay with it.

But instead, I found the article to be simply another a vile piece of teacher-bashing by the notorious Tribune Company. It's the kind of crap we have become used to reading here in the Chicago Tribune. In this case, questionable research methods based on the so-called "value-added" model, were used to supposedly rate teachers' effectiveness based entirely on student test scores. The Times faux researchers claimed that they "controlled" for outside-of-school factors, such as the effects of poverty, family and community issues, racism, etc... Of course they didn't, couldn't in a purely statistical study. To make matters worse, teachers' names and pictures were posted along with possible career-ending assumption made about them without any chance for them to respond. Talk about bullying!

But on the positive side, the Times piece inspired lots of us researchers, educators, bloggers and tweeters to get off our asses and respond which created some good controversy--not just about the Times piece, but about the limitations and proper use of value-added assessment.. Hopefully some light will be shed on this important issue and some valid and more authentic approaches to teacher evaluation will emerge. I also hope that, in the end, the Times will pay the price for its callous disregard for ethical research standards and its disrespect for the teaching profession.

I've collected some good tweets and links in response:

Larryferlazzo

New blog post "L.A. Times Prints Cheap Shot At Teachers" http://bit.ly/8YJhlE 

SchlFinance101

http://goo.gl/vxD6 LA Times "analysis" assumes prior scores correct entirely for other background characteristics? No details provided?

  shermandorn

Re L.A. Times story: my wonkish thoughts on growth models 5 years ago: http://shermandorn.com/wordpress/?p=416
DianeRavitch

Experts like Helen Ladd of Duke say that VAA is not ready for prime time, but LA Times doesn't care, ready to ruin careers.

leoniehaimson

dumb reductionist article at L.A Times using Value-added; admits should only be part of eval system but makes it all http://bit.ly/aXlTuZ

skrashen

LATimes publishes naïve analysis of teacher quality, with reporters as expert classroom observers=practicing educ research without a license 
 
 Larryferlazzo

New blog post "More On The L.A. Times Article" http://bit.ly/bm0wBy

DianeRavitch

New study by US Dept of Ed shows huge error rates when using VAA: http://tinyurl.com/2fn93uw  

DianeRavitch

I felt terrible for the LA teacher whose picture was in LATimes, id as ineffective by their measure. Why shd he be humiliated?  

charliemax

Union leader calls on L.A. teachers to boycott LA Times for reporting failing teachers | The Daily Caller http://goo.gl/nIJx  


More to follow