01/29/2009 10:39 am ET Updated May 25, 2011

George Bishop's and David Moore's 2009 Top Ten "Dubious Polling" Awards

My colleague at the University of Cincinnati, George Bishop, and I have launched what we expect to be an annual listing of the Top Ten "Dubious Polling" reports for the previous year. Posted on Stinky Journalism.Org, we intend this as a satirical look at some of the practices of the major media pollsters.


As the opening paragraph notes: "Every year, poll watchers are confronted with poll results and commentary that defy either logic or science, often raising question about the very utility of polls. Typically, the problems are not with the method of conducting polls, but with the pollsters themselves - as they focus on what they believe is entertaining and appealing to the audience rather than an accurate reflection of public opinion. In the process, pollsters manipulate public opinion or write commentary that makes a mockery of what the public is really thinking."


Each award is ranked, from a low of one set of crossed fingers to a high of five sets. Pollsters generally know in their hearts when all is not right with their polls, but they (figuratively) cross their fingers and hope that no one notices anything amiss. The five crossed-fingers icon is the ultimate in wishful thinking, perhaps the equivalent of football's "Hail Mary pass" for the truly untrustworthy poll.


Our top award - earning the five crossed fingers - goes to all the major media polls[1] for their prediction of Giuliani as the early Republican frontrunner. Collectively this group, beginning more than one year prior to the first statewide electoral contest in Iowa, relentlessly, and without regard for any semblance of political reality, portrayed Rudy Giuliani as the dominant Republican candidate in a fictitious national primary.


Other "Dubious Polling" awards are:

·        Loopiest Poll Award: Pew Research Poll, for weekly pre-election polls in October that showed wild swings in Obama's lead.

·        Shooting Yourself in the Foot Award: Gallup Poll, for publishing two polls on Feb. 25, 2008, that contradicted each other.

·        Over-the-Top Gloating Award: Gary Langer, polling director of ABC, for writing that "What I liked best about the final New Hampshire pre-election polls [which erroneously predicted Obama to win] is that I didn't do any of them" - cleverly completing his polling in the Granite State far enough away from the election to avoid having his results compared with the election outcome.

·        180 Degree Award: CBS News/New York Times and USA Today/Gallup polls, for coming to opposite conclusions about the controversy over Rev. Jeremiah Wright.

·        Waiting for Godot Award: The American Association for Public Opinion Research Committee that still has not issued a report on the erroneous predictions in the N.H. Democratic Primary.

·        Who Knows? Award: Pew Research, ABC News/Washington Post, and Los Angeles Times/Bloomberg polls for contradictory conclusions about public support for Wall Street bailout.

·        Wake-Me-Up-When-It's-Over Award: NPR, Kaiser Family Foundation, and the Harvard School of Public Health Survey for a vague 131-word question.

·        Flip-Flop Award: CNN for two December polls that showed opposite results of the public's support for the auto bailout.

·        For Sale! Award: Peter D. Hart Research Associates, for their General Motors-sponsored poll that found (surprise! surprise!) overwhelming public support of auto industry bailout.


For a full description and rationale for the awards, go to Stinky Journalism.Org.

[1] These include polls by the Associated Press, ABC News/Washington Post, CBS News/New York Times, CNN, FOX, NBC News/Wall Street Journal, USA Today/Gallup, Newsweek, the Los Angeles Times/Bloomberg, and Pew Research.