The Wire's David Simon: "Statistics will always lie" when someone's job is at stake

Via The Dish, an interview with Bill Moyers in which Wire creator David Simon highlights the perverse effect of performance incentives:

One of the themes of The Wire really was that statistics will always lie. Statistics can be made to say anything. You show me anything that depicts institutional progress in America: school test scores, crime stats, arrest reports, anything that a politician can run on, anything that somebody can get a promotion on, and as soon as you invent that statistical category, fifty people in that institution will be at work trying to figure out a way to make it look as if progress is actually occurring when actually no progress is. I mean, our entire economic structure fell behind the idea that these mortgage-backed securities were actually valuable, and they had absolutely no value. They were toxic. And yet they were being traded and being hurled about, because somebody could make some short-term profit. In the same way that a police commissioner or a deputy commissioner can get promoted, and a major can become a colonel, and an assistant school superintendent can become a school superintendent, if they make it look like the kids are learning and that they’re solving crime. That was a front-row seat for me as a reporter, getting to figure out how once they got done with them the crime stats actually didn’t represent anything.

I can't help but note that when the Michelle Rhee testing scandal broke in late March, I used an episode of The Wire to illustrate Campbell's Law -- the principle that incentives corrupt:

A city councilman who's trying to get the police commissioner to dance to his tune flays him and the mayor at a public meeting for high crime stats. Under pressure from the mayor, the commissioner promises to reduce felonies by 5% and keep the year's murder count under 275. The commissioner and deputy commissioner in turn lay the wood to their subordinates. Here's Rawls, the dickish deputy (my rough transcript):

You will reduce the felonies by 5% or more or...let no man come back alive.  In addition, we will hold this year's murders to 275 or less...[there's] no excuse I will accept. I don't care how you do it. Just fuckin' do it.

A world-weary police major named Colvin, just six months away from retirement at the 30-year mark, translates that for us:

We all understand there are certain...processes by which you can reduce overall felonies. You can, ah, reclassify an ag assault or you can unfound a robbery...but, ah, how do you make a body disappear?

 Somehow, it would seem. The commissioner concludes, unfazed:

If you want to continue wearing these oak clusters you will shut up and step up. Anyone who can't bring in the numbers will be replaced by someone who can.

 In this week's Ethicist feature in the New York Times Magazine, a further illustration:

I am a high-school student. Recently my school came up for review. Members of an accreditation board walked around the school for a few days, observing classes and asking students questions. Accreditation reviewers pulled students out of the crowd to speak at length. This was a huge deal for the administration. To prepare, many of my teachers began coaching us. They told us what to say if one of the accreditation reviewers asked us about certain education standards and asked us not to say anything negative about our school. 
 
I felt uncomfortable during all of this. By coaching us, aren’t our teachers subverting the inspection? Or is it expected that preparations should be made? NAME WITHHELD, LA JOLLA, CALIF.

A caveat, though. While it's always necessary to be skeptical and vigilant about performance measures, we can't do without them. Unemployment can be gamed. GDP can be gamed. Student performance can be gamed. Crime statistics can be gamed.  Accreditation inspections can be gamed.

Of course.  But in a society that has at least partially effective mechanisms for holding public officials accountable -- the media, competing officials, voters, courts -- some measure of truth will out over time. Crime really has dropped dramatically over the past two decades -- if not in the Baltimore of The Wire, in the nation's cities taken together.  GDP growth measures may mask some economic distortions, but no one in the U.S. is pretending right now that, say, the annual growth rate is 6%.  Nixon may have slapped on wage and price controls to win reelection, but most presidents will make some attempt to balance the short- and long-term economic effects of their decisions.

Tying individuals' pay directly to "performance" is dangerous and often counterproductive-- to varying degrees in different fields.  But we can't do without performance measures.

There is much more to the interview than this point about incentives -- the total futility of the drug war, the danger to democracy coming from the erosion of local reporting (which Simon sees as a key antidote to the kind of performance-fixing he sees as inevitable), the growth of oligarchical control of American institutions (though he does not really suggest that oligarchy was ever less pervasive in the U.S.). But "statistics will lie" is at the center of Simon's vision of how power corrupts. Read the whole thing.


As an aside, in his headnote to the interview Moyers rather fulsomely compares David Simon at length to Charles Dickens. That's a disservice to Simon. He has none of Dickens' mawkishness.

More on The Wire
The dignity, the intensity, the concentration in the most mundane human tasks
The Baltimoring of America
Krugman, The Wire beat you to it
Death of literature, greatly exaggerated
Pay for performance, Baltimore police style