Score One for the Polls

The white-knuckle moments on election night were fewer than many people expected. One source of fear had been the suspicion that the polls were wrong. Even after the networks started projecting winners, people with memories that reach back to 2000 were still sweating (after all, ABC projected Pennsylvania with no precincts reporting).

There were the usual bad reasons to doubt polls: skepticism about the theory of probability (how can 600 people represent a million?), unreasonable suspicion of social scientists (we're all biased), and the psychological power of anecdotes (no one has ever polled me, so how could they be accurate?).

But there were also some good reasons to be skeptical: the definition of "likely" voters, the possibility of missing young voters without landlines, the tendency of racists to pre-cover their tracks. We also had the bad history of exit poll errors from 2004. And the reasonable fear that, whatever voters' intentions, they wouldn't be captured by our highly fallible (to put it charitably) system of collecting votes.

Yet when it was over, the polls were basically right. I compared the final non-partisan poll averages from RealClearPolitics.com with the near-final results reported the day after the election.

In 26 out of 37 states, the poll average was within 3 percentage points of the final outcome, within the reported margins of error. The average for the states was an error of 2.7%. Not bad for surveys of just a few thousand people, attempting to predict the future behavior of tens of millions of people, acting in secret, in an emotionally-charged election, after a long process that had often defied prediction.

That average error does include some big misses, but they all occurred in states that weren't close (and where polling was not as intensive), such as Alaska and Arkansas. Overall, the polls were off by 6 points or more in just four states, and those were all decided by 9 points or more on election night, so it didn't much matter.

And in the close states the polls were very good. Of the eight states where the final vote was within 5 percentage points, the poll average was accurate to within 2 points in all but one (Indiana, where the poll average predicted a close Obama loss).

This doesn't mean we shouldn't be skeptical of polls (or social scientists - because we are mostly liberals, or worse). But in national elections there are lots of polls that can be averaged. That reduces errors due to chance, as well as differences in polling methods. And there is a lot of public scrutiny, which serves to police bad polling services.

Skepticism about science is good. But instead of turning us off scientific information altogether, skepticism should lead to its more careful consumption.

Read more reaction from HuffPost bloggers to Barack Obama's victory in the 2008 presidential election

Comments

[Oregon Ivy]

Nate at five-thirty-eight had it spot on, right down to every state.

Thank G0d for pollsters and the internet, because I believe it kept the Rethugs from stealing another election this time around (using r. a. c. e. as the excuse for the "small percentage" of error). However, with pollsters predicting such wide margins, they couldn't get away with it again without the populace ...er... taking matters into their own hands.

[sifta]

I think that the bar graph is screwy. The red bars on either side are both 4+ points worse? There's a typo!

[Philip N. Cohen - Huffpost Blogger]

Thanks, sifta. Fixed it. -pnc