Polling the New Hampshire Primaries: What Happened?

There was no shortage of polls going into the New Hampshire primary in 2008 and it looks like we all missed the mark on the Democratic side. This will require a lot of scrutiny in the coming days and weeks, but here are some initial thoughts on what has been happening:

1. According to the exit polls, 18% of the voters said that they made up their minds on primary day. That is just an unprecedented number. I have polled many races, especially close ones, where 4% to 8% have said they finally decided on their vote the day of the election and that can wreak havoc on those of us who are in the business of capturing pre-election movements and trends. But nearly one in five this time?

2. It looks like the always feisty voters in both Iowa and New Hampshire have rejected pre-election coronations. In the case of Iowa, Democratic voters said that Mrs. Clinton is not inevitable, while in New Hampshire they were not ready to endorse the Obama train without checking the engine.

3. The compressed schedule of the two events may have had an impact. Normally the winning candidate gets an initial big bounce out of Iowa, and then plateaus. Then the next primary race begins. With less than five full days, Obama got his bounce in New Hampshire, then the settling down period began on the last day -- under the radar screen.

4. My polling showed Clinton doing well on the late Sunday night and all day Monday -- she was in a 2-point race in that portion of the polling. But since our methods call for a three-day rolling average, we had to legitimately factor the huge Obama numbers on Friday and Saturday -- thus his 12 point average lead. Unfortunately, one day or a day-and-a-half does not make a trend and we ran out of time.

5. Going into the New Hampshire primary, we certainly did see Clinton holding on to a significant lead among women and older voters. But we were focusing on Obama's massive lead among younger and independent voters. We seem to have missed the huge turnout of older women that apparently put Clinton over the top.

6. We expected that Obama would receive the lion's share of independents and drain the Republican primary of these voters. It now appears that, perhaps with a sense that Obama had a lock on the Democratic side, independents felt free to vote on the Republican side and reward their hero, John McCain.

We will pour through the data and try to come up with something more definitive, but those are my early observations. There is much speculation that Senator Clinton's crying incident may have offered voters -- especially women -- a peek at the human side of someone who is often seen as scripted. I think she also scored points during the ABC debate Saturday night when she declared, amid a discussion about the country's desire for a change in direction, that electing a woman would represent a big change in itself. Her numbers did go up in that last 24-hour period.

On the other side, most of us did a whole lot better coming close to the numbers on the Republican side of the aisle. But this is one of those cases that remind us that pre-election polls are guides to voter attitudes and shifts. All things considered in this and other cases, we pollsters still do a creditable job.

Comments

[JMorgan]

This doesn't explain the exit polls which, according to news people, showed Obama with the same large lead.

[kevinabt]

This election is rigged. It is a fraud. The voting system is compromised and the results do not reflect the votes that were cast. 75% of NH uses electronic voting that has no paper trail. Votes are stored electronically and can be changed without anyone knowing it. The programmer of the voting machine (or someone who has altered the original progam) determines the outcome. The exit polls are correct, the official count is a lie. The voting machines changed the votes to give Clinton the win.
The other 25% of NH that doesn't use electronic machines is also compromised. There is definative proof. In Sutton county 31 people voted for Ron Paul, the official tally was reported as 0 votes for him. The person running the counting operation had to admit to falsifying the numbers when the voters confronted them.
This is election fraud people. We have to wake up. Without a fair vote we dont live in a democracy where elected officials run government. We live in tyranny where a small group of people appoint the officials with no accountability to the people of the country.

[iswearington]

i remember the good old days...whe­n exit polls and the calculus of statisticians were law....and would trigger recounts..­.now after george bush s precedence setting thefts of two elections.­... they are considered conspiracy theorys.el­ves...dwar­ves...and men! we march to battle! we will never live under the rule of orcs and nazgul mounted witch kings..

[CoffeeBreak]

It's about race-gap polling! This article from NYT is the best article about it yet. Gives the race-gap polling theory the clearest and most detailed explanation I've seen, and it's by a guy who worked for gallup. Here is the key, and pretty culprit-finding to me, paragraph:

"Poorer, less well-educated white people refuse surveys more often than affluent, better-educated whites. Polls generally adjust their samples for this tendency. But here's the problem: these whites who do not respond to surveys tend to have more unfavorable views of blacks than respondents who do the interviews­."

Full article:

http://www.nytimes.com/2008/01/10/opinion/10kohut.html?ex=1357707600&en=035df84517b516b7&ei=5088&partner

[strangelet]

Zogby --

Nice shot at suppressing comment.

All of you pollsters have been running on the edge for a couple of elections now. In close elections, the phrasing of the questions assumes extraordinary importance. I also believe (purely from personal experience) that polls conducted close to, or on, the election day tend to elicit responses that may reflect the interviewee's annoyance more than his or her actual feelings.

While the concept of political polls is unassailable (any candidate wants a snapshot of his or her popularity), the notion of continuing polling right up to the brink -- or beyond, in the case of exit polls -- is more problematic.

Who benefits from last second polls? Not any candidate, because it is too late to do anything about it. Not the public in general, because the electoral results will soon reveal the true choice of the people. The only possible beneficiaries are the pollsters themselves, who can show how their predictions are borne out by reality.

How inconvenient when even the last polls prove to be wrong. Time for some serious 'splainin'. Which you are doing in your article. How about an argument for moderation? Stop polling a day before the actual election. Let the undecideds make up their minds on their own. I guess that voluntary exit polls provide useful information to campaigns, but how 'bout you hold the results for a day, so people can just absorb the actual outcome at the polls before having to confront detailed analysis of why it happened that way.

Nobody really cares that you were wrong. They will still give as much credence to future polls as they did to the last one (that credence ranges from 0 to 100%). I am not being facetious when I say that all pollsters ought to ease up a little, before people start voting to prove the polls wrong. That would be a real perversion of democracy.

[Maxbyte]

John: I listened to Arianna on Larry King tonight and realize that, at least from her perspective, she feels victimized by publishing such erroneous polls, and even told Larry King there should be an investigation.

I agree with each of you about some inherent problems in this new primary schedule nationwide. A very common and frustrating problem for websites that are not necessarily data-oriented is that a LOT of the polls did not include critical elements everyone requires to decide if the polling results are useful.

With a reasonably strong data background, my first instinct is to look at the "N" [number of respondents], the margin of error, and how much methodology is openly provided.

What I found with a huge number of polls was that the N was 500 or less across all respondents. Information about the randomness of the samples, or any other methodological fundamentals was often missing. The error of the mean [margin of error] in a lot of those small polls was probably 4% to 7%.

What journalists do not necessarily understand is that there are immediate problems with such small samples. The most glaring tends to be that a margin of error of 5% or so applies only when all respondents are included in the analyses. That is rare. In almost every instance, the analyses were within the Democratic and Republican groups, or within the Democratic, Republican, and Independent groups.

Of course, we both know that breaking up small samples in that manner dramatically increases the sub-group margin of error.

There were a number of properly conducted polls, including Zogby, Reuters, CBS, ABC, Pew, and Rasmussen, most of which were well documented.

The fact that even the properly conducted polls appear to have failed to do anything at all with the huge number of "may change" and undecided voters. That is what appears to have turned the race at the last moment.

I agree with Arianna again that all of the polls should have provided more information about how they were conducted.

[navalvet]

"Unfortunately, one day or a day-and-a-half does not make a trend and we ran out of time."
That may be true of political polling but it is patently untrue of politics.

[jimsey]

Hm. Has anyone considered that pollsters generally do a good job and that perhaps a good faith error was made here and failed for the reasons stated (among others yet to be determined). Or is a non-conspiracy theory take on this just too boring?

[Fightnmad]

5. "Going into the New Hampshire primary, we certainly did see Clinton holding on to a significant lead among women and older voters. But we were focusing on Obama's massive lead among younger and independent voters. We seem to have missed the huge turnout of older women that apparently put Clinton over the top."

Scientifically, and not politically speaking, the last sentence is particularly bothersome. How did you or ANY pollster get "the huge turnout of older women"? I don't recall any ballot where height, weight, sex, or age is etched in code, or ANY reliable source that states older women are apt to vote for a woman.

This seems to be pure conjecture and a disturbing underlying assumption perpetrated by you and MSM in the aftershock of this phenomenon. If you can't believe your ears and eyes, shouldn't that be an indicator for further scientific inquiry on the elephant in the living room?

My skepticism meter is hitting the red zone, and with no offense meant to the voters in NH, it's unlikely these people voted to "send a message" to punish the MSM for their raunchy treatment of Sen. Clinton, as some pundits claim. If the majority voted in support of Sen. Clinton, then what does that tell you about ALL of the incorrect predicting polls? Notice how accurate the primary polls were in regards to each of the Republican candidates, but off only in the Democratic numbers. Raise any red flags?

Did anyone do exit polling in NH to arrive at this unusual conclusion?

Warning: America stopped believing in exit polls after the 2000 presidential election, and the SNAFU follow-up in 04 for the very same reasons.

This could be a fluke, but if not, ask yourselves: "How likely is it, that we could be wrong so many times in the same election year?" Occam's Razor would answer, "Not very".

[chumleypotts]

But all the polls said Obama will win NH. He should demand a recount at the very least, or possibly request a congressional investigation.

[Grrr]

Just Google " The Bradley Factor" for one possible and plausible explanation. Then ask black candidates why they always feel they need 5+ points in any poll to expect an even finish. White voters lie. If this seems like conspiracy theory, then explain why all the Republican polls were dead on and all the Democratic ones were not. Additionally, of those undecideds who chose a candidate in the last hours, exit pollsters now say they broke about equally for Obama and Clinton, disproving the theory that Clinton's show of emotion held inordinate sway..

[RichLiberal]

All one needs to do is google around a bit for zogby, bias, polls, Clinton etc and one will find that he has systematically been undercounting Hillary for months. I found it particularly amusing that he hit Edwards right on the nose at 17%.

So, what went wrong with the polls? What went wrong was that Zogby's scheme to manipulate the data blew up in his face. Why the media even reports this guy's propaganda is beyond me.

[Maxbyte]

I would posit that the leading reason Zogby and other pollsters were on the wrong planet prior to the New Hampshire primary is that they believed the Iowa outcome was credible. It wasn't. Never has been.

The Iowa Caucuses are not representative of Iowans, but they are always held up by the media as a meaningful event. They should be outlawed.

A lot of New Hampshirites apparently believed at first that Iowans may have pointed the way. Then they listened to Obama and heard nothing other than supercilious nonsense.

Obama is an excellent orator. But, to a limited extent, Obama is the Democratic version of George W. Bush: he uses the flag, patriotic slogans, and other symbolic euphemisms to avoid substantive discussion.

Beyond that, Obama has nothing in common with Bush. But Obama is going to have to stop skipping over and around substance.

[lawyeredup]

Nonsense! The media was trying to knock Hillary out with a one two puch in favor of Obama whom as Hillary said has been getting a free ride from the press while everything she does or say is analyzed and critised.
Hillary was right when she says electing her is change in itself -- and women voter should wake the hell up!

[reactionrynihilist]

How come the MSM doesn't investigat Diebold? Remember the hit job against GM when they put bottle rockets on the gas tanks of trucks to make it look like they blew up on impact?

From the little I can dig out of the media about Diebold I am glad that my district still has the old fashioned machines. We spent billions to bring democracy to Iraq, yet we can't spend a few million to upgrade to machines that can count votes and provide a paper trail here?

This is a scandal and is getting no coverage. I am not going to hold my nose and vote for a democrat, I am going to throw away my vote on third parties that represent my views and I won't respect an incumbant democrat who doesn't secure our votes.