Saturday, March 30, 2013

Guess what happens when you tie teachers' (and administrators') pay directly to student test scores?

Ms. Parks admitted to Mr. Hyde that she was one of seven teachers — nicknamed “the chosen” — who sat in a locked windowless room every afternoon during the week of state testing, raising students’ scores by erasing wrong answers and making them right. She then agreed to wear a hidden electronic wire to school, and for weeks she secretly recorded the conversations of her fellow teachers for Mr. Hyde. 

In the two and a half years since, the state’s investigation reached from Ms. Parks’s third-grade classroom all the way to the district superintendent at the time, Beverly L. Hall, who was one of 35 Atlanta educators indicted Friday by a Fulton County grand jury. 
Move over, Michelle Rhee (test cheating was also apparently rife in the D.C. schools under her tenure).  So often it's the icons of toughness, the lionized apparent miracle workers who create a culture of fear and greed that breeds corruption:

“The cheating had been going on so long,” Ms. Parks said. “We considered it part of our jobs.”
She said teachers were under constant pressure from principals who feared they would be fired if they did not meet the testing targets set by the superintendent. 

Dr. Hall was known to rule by fear. She gave principals three years to meet their testing goals. Few did; in her decade as superintendent, she replaced 90 percent of the principals. 

Teachers and principals whose students had high test scores received tenure and thousands of dollars in performance bonuses. Otherwise, as one teacher explained, it was “low score out the door.”
The perverse incentives go all the way up the food chain and have been exacerbated by the Obama administration:
During the decade [superintendent Beverly L. Hall, now indicted] led the district of 52,000 children, many of them poor and African-American, Atlanta students often outperformed wealthier suburban districts on state tests. 

Those test scores brought her fame — in 2009, the American Association of School Administrators named her superintendent of the year and Arne Duncan, the secretary of education, hosted her at the White House. 

And fortune — she earned more than $500,000 in performance bonuses while superintendent.
I am not presuming Beverly Hall's guilt, But it seems beyond question that she created a climate conducive to corruption, whether or not she authorized it.

We can't do without statistical measures of performance in government and organizations generally. Yet they are powerful agents of corruption. Pleae indulge a repost from 4/16/11:

The Wire's David Simon: "Statistics will always lie" when someone's job is at stake

Via The Dish, an interview with Bill Moyers in which Wire creator David Simon highlights the perverse effect of performance incentives:
One of the themes of The Wire really was that statistics will always lie. Statistics can be made to say anything. You show me anything that depicts institutional progress in America: school test scores, crime stats, arrest reports, anything that a politician can run on, anything that somebody can get a promotion on, and as soon as you invent that statistical category, fifty people in that institution will be at work trying to figure out a way to make it look as if progress is actually occurring when actually no progress is. I mean, our entire economic structure fell behind the idea that these mortgage-backed securities were actually valuable, and they had absolutely no value. They were toxic. And yet they were being traded and being hurled about, because somebody could make some short-term profit. In the same way that a police commissioner or a deputy commissioner can get promoted, and a major can become a colonel, and an assistant school superintendent can become a school superintendent, if they make it look like the kids are learning and that they’re solving crime. That was a front-row seat for me as a reporter, getting to figure out how once they got done with them the crime stats actually didn’t represent anything.
I can't help but note that when the Michelle Rhee testing scandal broke in late March, I used an episode of The Wire to illustrate Campbell's Law -- the principle that incentives corrupt:
A city councilman who's trying to get the police commissioner to dance to his tune flays him and the mayor at a public meeting for high crime stats. Under pressure from the mayor, the commissioner promises to reduce felonies by 5% and keep the year's murder count under 275. The commissioner and deputy commissioner in turn lay the wood to their subordinates. Here's Rawls, the dickish deputy (my rough transcript):
You will reduce the felonies by 5% or more or...let no man come back alive.  In addition, we will hold this year's murders to 275 or less...[there's] no excuse I will accept. I don't care how you do it. Just fuckin' do it.
A world-weary police major named Colvin, just six months away from retirement at the 30-year mark, translates that for us:
We all understand there are certain...processes by which you can reduce overall felonies. You can, ah, reclassify an ag assault or you can unfound a robbery...but, ah, how do you make a body disappear?
 Somehow, it would seem. The commissioner concludes, unfazed:
If you want to continue wearing these oak clusters you will shut up and step up. Anyone who can't bring in the numbers will be replaced by someone who can.
 In this week's Ethicist feature in the New York Times Magazine, a further illustration:
I am a high-school student. Recently my school came up for review. Members of an accreditation board walked around the school for a few days, observing classes and asking students questions. Accreditation reviewers pulled students out of the crowd to speak at length. This was a huge deal for the administration. To prepare, many of my teachers began coaching us. They told us what to say if one of the accreditation reviewers asked us about certain education standards and asked us not to say anything negative about our school. 
I felt uncomfortable during all of this. By coaching us, aren’t our teachers subverting the inspection? Or is it expected that preparations should be made? NAME WITHHELD, LA JOLLA, CALIF.
A caveat, though. While it's always necessary to be skeptical and vigilant about performance measures, we can't do without them. Unemployment can be gamed. GDP can be gamed. Student performance can be gamed. Crime statistics can be gamed.  Accreditation inspections can be gamed.

Of course.  But in a society that has at least partially effective mechanisms for holding public officials accountable -- the media, competing officials, voters, courts -- some measure of truth will out over time. Crime really has dropped dramatically over the past two decades -- if not in the Baltimore of The Wire, in the nation's cities taken together.  GDP growth measures may mask some economic distortions, but no one in the U.S. is pretending right now that, say, the annual growth rate is 6%.  Nixon may have slapped on wage and price controls to win reelection, but most presidents will make some attempt to balance the short- and long-term economic effects of their decisions.

Tying individuals' pay directly to "performance" is dangerous and often counterproductive-- to varying degrees in different fields.  But we can't do without performance measures.

There is much more to the interview than this point about incentives -- the total futility of the drug war, the danger to democracy coming from the erosion of local reporting (which Simon sees as a key antidote to the kind of performance-fixing he sees as inevitable), the growth of oligarchical control of American institutions (though he does not really suggest that oligarchy was ever less pervasive in the U.S.). But "statistics will lie" is at the center of Simon's vision of how power corrupts. Read the whole thing.

As an aside, in his headnote to the interview Moyers rather fulsomely compares David Simon at length to Charles Dickens. That's a disservice to Simon. He has none of Dickens' mawkishness.

No comments:

Post a Comment