Pre-Election Polling in New Hampshire: What Went Wrong?

In the wake of the New Hampshire primary, much press coverage has been focused on the pre-election polls, in particular on the Democratic presidential primary. Headlines indicate that pre-election polls were misleading or wrong. Yes, all of the pre-election polls showed Senator Obama ahead in the final pre-election polls. Clearly, on this count, they all failed to reflect the eventual outcome. But the polls also were surprisingly accurate in measuring support for candidates other than Senator Clinton --with estimates of around 36% for Senator Obama, 19% for Senator Edwards, and 6% for Governor Richardson (compared to final estimates of 36%, 17% and 5%, respectively). They went astray in the case of Senator Clinton's final vote.

The final pre-election poll estimates reinforce several points:

• Polling is a scientific process that attempts to capture information about individual attitudes and behaviors, both of which are subject to variation over time. Events following the conduct of a survey or poll can result in opinion and behavior changes.

• Polls and surveys are subject to multiple sources of error--including failure to sample all the voters and social desirability bias just to mention two.

• The role of undecideds in a close election is difficult to understand in advance. As late as Monday, January 7, polls indicated that up to 10% of Democratic voters were still undecided and the CBS News Polls cited that "28% of Democratic voters say their minds could still change."

• Understanding the methodology related to the conduct of the poll, the allocation of undecideds, and the likely voter models becomes increasingly important when elections are close.

• All polls are subject to effects due to missing some randomly selected
respondents because they are not at home when called, refuse to be interviewed, or are unavailable for other reasons. In most past election polling, this problem has not appeared to affect estimates appreciably. However, there is always the possibility that an effect may occur in a particular election.

The forces shaping the discrepancies between the pre-election polls and the actual outcomes in New Hampshire deserve immediate and thorough examination and analysis, if we are to understand what happened there and apply that understanding to state primaries to follow. The American Association for Public Opinion Research (AAPOR) supports the disclosure of polling methodology--and as advocated by www.pollster.com -- including the disclosure of information related to questions used in the poll, sample size, response rates, as well as the likely voter models and undecided allocations used by the pollster. Only when the data are fully available to scholars of pre-election polls will we understand the effects of alternative models and design, as well as the potential impact of any bias, on pre-election poll estimates.

AAPOR strongly supports the recommendation made by Gary Langer of ABC News (and an AAPOR member) when he advocated for the producers of the New Hampshire pre-election polls "to look at the data, and to look at it closely, and to do it without prejudging." Clearly, the NH pre-election polls warrant more analysis and research before we attempt to draw even tentative conclusions.

[Note: Public Opinion Quarterly, the journal of the American Association for Public Opinion Research, has published three articles that address some causes of error in self-reports of voting or vote intention in races involving minorities and women. They do not "explain" what happened in NH, but they offer some background for considering poll results in that primary and ones
to come. The articles are available on the AAPOR web site.]

Comments

[jeffrupnick]

All of this is well and good, but it is hard for me to accept that all of these reasons just happened to magically take place at the exact moment to overcome what polls predicted would be a double-digit victory for Obama. Maybe, rather than searching for reasons why these polls were wrong, we should be looking at this from a different perspective, and look at the actual votes cast for Clinton, and scrutinize those. I hate to be like this, but it seems as though something fishy is going on here, and I don't like it one bit.

[Jane]

Also, the polls might not reflect subsequent fiddling with the voting machines.

[argeec]

How about an article explaining why we shouldn't believe that the polls were right and the final vote was jacked in Hillary's favor?

[MPeter]

The poll was rigged and now they are using her fake tears to explain the fraud. Here is a snippet:
The Diebald Voting Machines miscount votes and precients and hand count shows Sen. Obama actually won in New Hampshire.

Diebold favors Hillary, hand count for Obama
Wed, 01/09/2008 - 05:46 - clark

I used the Comma delimited database: NH municipalities hand count vs use Diebold machines from BlackBoxVoting.org to see if there was a deviation between the results from precincts which used hand counts and those which relied on Diebold machines. The results were astonishing. :

Updated: 5:05 AM (EST) - Results tallied for 209 out of 236 of the municipalities.


By Percentage
Method Hillary Clinton Barack Obama
Diebold Machines 53.23% 46.77%
Hand Count 47.47% 52.53%


By Votes
Method Hillary Clinton Barack Obama
Diebold Machines 82860 72807
Hand Count 18898 20912


By Number of Municipalities Won
Method Hillary Clinton Barack Obama
Diebold Machines 54 33
Hand Count 43 77


About 81% of the votes will be "counted" by the Diebold machines