Friday, November 6, 2009

Lies, Damned Lies and Statistics

This struck me as an example of how statistics can be used to reach any conclusion, especially if we don't pay close attention. Yesterday's Associated Press had an AP Impact story about the missteps of the Cash for Clunkers program. Titled Clunker Pickups Traded for New Pickups, the scandalous exposé reveals some dirty laundry from the now-discontinued government stimulus program. In particular, the report focuses on transactions where the new vehicle purchased got the same or only marginally better gas mileage (and, in a few cases, worse mileage) than the "clunker" being traded in.

The most common deals under the government's $3 billion Cash for Clunkers program, aimed at putting more fuel-efficient cars on the road, replaced old Ford or Chevrolet pickups with new ones that got only marginally better gas mileage, according to an analysis of new federal data by The Associated Press.

The opening statement "The most common deals…" implies that most of the transactions of the program failed to yield an improvement in fuel mileage, one of the intentions of the program (in addition to trying to boost the economy and assist the ailing auto industry). But another report, this one citing a Washington Post summary of the program, states: "The average clunker traded-in, the Post reports, had an EPA fuel economy rating of 15.8 miles per gallon. The average new car purchased, 24.9 - an improvement of 9.1 miles per gallon."

Going back to the AP Impact report, the press release continues:

In scores of deals, the government reported spending a total of $562,500 in rebates for new cars and trucks that got worse or the same mileage as the trade-ins — in apparent violation of the program's requirements. The government said it is investigating those reports and said in some cases they were probably entered incorrectly by dealers or based on outdated fuel economy figures.

This is a program that involved some 690,000 transactions (according to US News and World Report) and nearly $3 billion in government disbursements. It seems to me that "scores" (a score being roughly 20) of transactions involving just over half a million dollars out of $2.88 billion is a fantastic success rate; it's the exception that proves the rule.

But the reporter has his teeth in the issue and will not be swayed:
The new data, obtained by the AP under the Freedom of Information Act, include details of 677,081 clunker trade-ins processed by the government through Oct. 16. More than 95,000 of the new vehicles purchased under the program — or about one in seven — got less than 20 mpg, according to the data.

Yeah, this hard-hitting investigation dug deep, having to invoke the Freedom of Information Act to get at the sordid truth! But to my almost innumerate brain, that sounds like six out of seven purchases were of cars getting more than 20 mpg. That sounds like success to me. And there is no mention of the composite mileage of the clunkers traded in on the sub-20-mpg cars. A 14 mpg truck traded on a 19 mpg truck would be something over a 30% improvement while still earning a berth on the AP blacklist.

I don't want to rush to support the yahoo who used the program to buy a new Hummer or who drives a big 4WD pickup for no apparent reason, but it seems short-sighted to assume that everybody trading a pickup for a new pickup is doing so for illegitimate reasons. My brother, for example, works in construction and needs a pickup. Had he purchased a new one under this program he would be among those lynched by this AP Impact report.

It's absolutely proper that the program be scrutinized and reported upon, and I'm pleased to hear that there is some government follow-up on those transactions that do not appear to meet the program guidelines. But this all seems very routine; meanwhile, the AP Impact report seems like an exercise in reverse-think, like a journalism school assignment where one is asked to take facts and make them convincingly say the opposite of what they conclude. It's reality TV in print form: a story specifically tailored to push people's angry buttons, a tactic that amounts to the core operating principle (if you can call it that) at Faux News.

We've had far too much fear- and hate-mongering masquerading as journalism; a misleading dispatch like this --from the AP of all sources--only panders to frothing and willfully misinformed troglodytes who already exert entirely too much influence over our news coverage.

(PS: I remember a similar-toned story in the tabloid USA Today from a decade ago. Showing a large graphic automobile fuel gauge with the needle on E, the headline shouted "Flying on Empty?" The story went on to tell--shockingly!--that in something like the preceding nine years 10 or 12 aircraft had "pushed back from the gate without enough fuel to get to their destination... and four of them took off!" No crashes, no fatalities, no injuries. Just a desperate Glenn-Beck-like attempt to sell papers by appealing to base fear and personal safety. I remember thinking that that statistic was, in reality, an amazing endorsement of the safety measures currently in place; few industries could boast such a performance--think of the number of departures in that time!--yet here was "The Nation's Newspaper" finding terror in a sunny day. I realized then that we have no reason to think they're doing a better job on subjects with which we are unfamiliar than the butchery I routinely see them perform on subjects where I DO know a thing or two. I've not purchased one since.)


Jeffy said...

Stats and reporters seems to be a dangerous mixture. Given their bent to want to blow things out of proportion, reams of data are a reporters best friend. They can have data to back up what they say, yet can say pretty much whatever they want.

I suspect that the 'analysis' of this data included a bogus cataloging of all of the specific trades - Ford F150 for Ford F150, Ford Explorer for Honda CR-V, Ford Explorer for Toyota Camry, and so on, and it would not surprise me that given the relatively small number of pickup models available, and the fact that pickup owners often need a pickup, the specific examples of trading a pickup for another pickup would be the most common. It may have been even more common for a pickup to be traded for a small SUV or sedan, but what does it matter either way?

It doesn't seem that the specific trades are all that relevant. The program was mainly an effort to goose auto sales. By building in a way to improve fuel efficiency in the process it helped make the expense a bit more palatable. In that regard it seems to have been a success, given the average MPG improvement across all the trades. As you mention, even a swap of a 12 MPG truck for a 14 MPG truck can be a very good thing. Over the course of an average 15K mile year this trade will save about 175 gallons of gas. A trade between an 18 MPG SUV and a 22 MPG sedan would save only 150 gallons. When the mileage is really bad a small improvement can be a very good thing.

My experiences with news reports involving things that I have direct knowledge of are similar to yours. The story in the paper seems so far from what I know to be the actual case that I am surprised, and I too can't help but think that many other news reports are just as far off the mark.

wunelle said...

He lives!

I think that business with the airplane fuel levels was kind of my last straw. I don't know much about much, but the things I do understand are always got subtly wrong (or worse) in the dumbed-down mainstream coverage. I don't know why I would expect anything different on their coverage of other topics. And then when one adds in the element of pointed bias and the efforts to attract the biggest crowd in pursuit of advertising dollars, there's not much left to hope for.

I give a bunch of money to NPR. At least I feel like I'm contributing to one of the last functional journalistic enterprises in the country. (I used to know someone who used a shortband radio to listen to the BBC--and not BBC America--for the same reason.)