Polling the New Hampshire Primaries: What Happened?

There was no shortage of polls going into the New Hampshire primary in 2008 and it looks like we all missed the mark on the Democratic side. This will require a lot of scrutiny in the coming days and weeks, but here are some initial thoughts on what has been happening:

1. According to the exit polls, 18% of the voters said that they made up their minds on primary day. That is just an unprecedented number. I have polled many races, especially close ones, where 4% to 8% have said they finally decided on their vote the day of the election and that can wreak havoc on those of us who are in the business of capturing pre-election movements and trends. But nearly one in five this time?

2. It looks like the always feisty voters in both Iowa and New Hampshire have rejected pre-election coronations. In the case of Iowa, Democratic voters said that Mrs. Clinton is not inevitable, while in New Hampshire they were not ready to endorse the Obama train without checking the engine.

3. The compressed schedule of the two events may have had an impact. Normally the winning candidate gets an initial big bounce out of Iowa, and then plateaus. Then the next primary race begins. With less than five full days, Obama got his bounce in New Hampshire, then the settling down period began on the last day -- under the radar screen.

4. My polling showed Clinton doing well on the late Sunday night and all day Monday -- she was in a 2-point race in that portion of the polling. But since our methods call for a three-day rolling average, we had to legitimately factor the huge Obama numbers on Friday and Saturday -- thus his 12 point average lead. Unfortunately, one day or a day-and-a-half does not make a trend and we ran out of time.

5. Going into the New Hampshire primary, we certainly did see Clinton holding on to a significant lead among women and older voters. But we were focusing on Obama's massive lead among younger and independent voters. We seem to have missed the huge turnout of older women that apparently put Clinton over the top.

6. We expected that Obama would receive the lion's share of independents and drain the Republican primary of these voters. It now appears that, perhaps with a sense that Obama had a lock on the Democratic side, independents felt free to vote on the Republican side and reward their hero, John McCain.

We will pour through the data and try to come up with something more definitive, but those are my early observations. There is much speculation that Senator Clinton's crying incident may have offered voters -- especially women -- a peek at the human side of someone who is often seen as scripted. I think she also scored points during the ABC debate Saturday night when she declared, amid a discussion about the country's desire for a change in direction, that electing a woman would represent a big change in itself. Her numbers did go up in that last 24-hour period.

On the other side, most of us did a whole lot better coming close to the numbers on the Republican side of the aisle. But this is one of those cases that remind us that pre-election polls are guides to voter attitudes and shifts. All things considered in this and other cases, we pollsters still do a creditable job.

Comments

[davidmwe]

Polls are BS, but than again, we all want them as a gauge. I do agree that we do away with media with an agenda.

[Researchguy66]

Oh, for God's sake people, comparing hand-counted precincts to machine counted precincts demonstrates exactly ZERO. Here's why?

Larger precincts machine count and smaller precincts hand count. People who live in larger precincts are likely to have different preferences from people who live in smaller precincts. Even if that wasn't true in this particular primary, there's no way to rule it out as an effect.

Finally, this IS NOT COMPARABLE to 2000 and 2004. In those elections, the final vote count disagreed with the exit polling, not the pre-election polling. There's a big difference. Last night, the final results agreed with the exit polling, which is why there were able to say before a single vote was counted (minus Dixon Notch) that the race was too close to call.

I like conspiracies, but I'm afraid this ain't one of them.

[SpindrainDotCom]

With this disturbing polling debacle, one has to wonder just WHO was being polled.

Were pollsters trying to produce skewed numbers by [maybe] focusing on voters under the age of 40? Or mainly college students?

Were some pollsters trying to dissuade Edwards and Clinton supporters from bothering to show up to vote, considering the blowout their candidates were expected to suffer?

And/or were pollsters aiming for unrealisitc numbers in an effort to nudge undecided voters to jump on the Obama wave - their "inevitable" primary winner?

Considering how the numbers with the Republicans were close to spot on... it's rather curious.

[MetricSystem]

Rigged? Wow, you people are grasping at straws. I could see this accusation being thrown at Rethugs but not a Democratic candidate. I mean, if Hillary wins it must be rigged, but if Obama had won it would have been democracy in action? I guess some of you have bought into the Rethug attack machine against the Clintons.

[Luc65]

The bottom line is poll with such low population sample and high error margin are meaningless. When you have so many different trends, how can you make any prognostic by polling 1,000 people or less?

With a higher population sample you could have found
out that many people where undecided and factor that in.

There is absolutely no excuses for giving the wrong information based on insufficient data. Are the pollsters lazy?

[askmeificare]

Save allot of time and money and do away with polls altogether. They serve no useful purpose other than to give the media thousands of hours of stories to waste our time with.

BTW: Looks like the polling people and the hurricane predictors took the same classes in college.

[ajax2]

If Zogby is correct that nearly 20% of voters decided on last day, than his pre-polling numbers are virtually worthless.

[Nyla785]

Stay focused, people. 3915 U.S. dead in Iraq now. That makes about 13 killed over the first 9 days of the new year. We're still losing people in Iraq. But thank goodnes 'the surge is working'--it makes it as if they're really not dead at all!

[nolandalla]

One thing that favored CLINTON was being listed first on the ballot. I believe most if not all precincts listed the candidates alphbetically. Stanford professor Jon A. Krosnick has estimated the impact of appearing high on the ballot at three percentage points or greater. The same logic applies to phone books and business listings. Look and see how many AAA, AAAAA, and AARDVARK bail bondsmen, plumbers, and towing companies are listed in your local phone directory.

Without getting too technical, one of the most serious flaws with common political survey methodology (like Zogby) is the failure to employ weighed responses. Soft support and hard support are weighed exactly the same in most polls. This means a hard-core volunteer working for CAMPAIGN A is measured the same as a pseudo-undecided voter who is leaning towards CANDIDATE B. When both persons are called, the pollster registers the result as 1-1. Never mind that CANDIDATE A is *sure* to get a vote while CANDIDATE B could *lose* the vote, either by the voter changing his mind or deciding not to vote.

I speculate that about 20 percent of OBAMA's voter were very soft and jumped onto the bandwagon immediately after Iowa. They did so with little political forethought nor evaluation. OBAMA's name was everywhere in New Hampshite. Late polls reflected the "Coca-Cola effect" which is to illicit the name of the most popular product when pressed to answer a question -- "What is your favorite soft drink?"

Furthermore, some of those who expressed a preference for OBAMA did not show up and vote. That created a false expectation. Meanwhile, CLINTON supporters who were now in the underdog role were mobilized and turned out in greater numbers than expected.

All of this could have been avoided had pollsters weighed voter responses. Using the standard 1-5 scale would have been enough to show that the CLINTON campaign had a lot of 4s and 5s in her camp. But OBAMA had a lot of the 1s, 2s, and 3s.

Nolan Dalla
Las Vegas, NV

[in4success]

http://www.nytimes.com/2008/01/07/opinion/07poundstone.html?em&ex=1199941200&en=9c87c614f4814fe2&ei=5087%0A

[in4success]

"With less than five full days, Obama got his bounce in New Hampshire, then the settling down period began on the last day -- under the radar screen."

man -- i've heard some bullshit excuses and explanations before but this one just about takes the "PLEASE REMAIN SEATED -- EVERYTHING IS NORMAL -- YOU ARE NOT IN ANY DANGER -- DON'T PAY ANY ATTENTION TO THAT MAN BEHIND THE CURTAIN" hill to new and fascinatingly creative heights.

jesus christ!

if we've been dumbed down and conditioned skillfully enough to believe that load of crap then we are certainly doomed and bin laden himself will probably enter the race and be the next president of the united states..

WHEW!

[NewAmericanCenturySucks]

Theory #1 requires the 18% to break strongly for Clinton. The same exit polling you reference indicates that they did not.

Evidence for Theories #2, 3 or 4 should show up in either exit polling or hand-counted ballots. It did not.

Theories #5 and #6 are certainly plausible, but are presented without a shred of statistical evidence. Please present evidence of having polled disproportionately fewer older women / independents than actually voted, or withdraw these theories.

I submit for your consideration Theory #7:

Hand-counted: Obama 39 Clinton 35
Diebold-counted: Clinton 40 Obama 36

Between 8 and 9pm EST, I noticed the CNN vote counter incremented oddly (up for one candidate, down/zero for the other) three times. In all three cases, Clinton was helped, and Obama was hurt.

I am open-minded enough to consider the possibility of a third-variable effect (older women prefer to live in Diebold-intensive areas, e.g.). But until I am presented with hard statistical evidence to explain this effect, I remain thoroughly unconvinced that America is still a democracy.

[emsique]

God, I'm already sick of polls, talking heads, MSM,the issue free beauty contest the media has turned this into (not just the MSM!), and general American ignorance!
The candidates with the best credentials, Biden, Richardson and Kucinich were ignored and marginalized by the media and the public until now we have a horse race between a junior senator and a senator with one and a half terms. WTF?????!!!!

[in4success]

If the vote is already rigged, the idea is to feed the people a vaguely plausible explanation of how an ‘unexpected’ and completely ridiculous result could come about.

It’s all theater.

And the plot stinks.

And WE THE PEOPLE don't demand the votes are legit -- we just keep believing whatever side of the bullshit pile supports our self-indulgent, narcissistic views and emotions. we are conditioned well as consumers without the ability to see clearly, intentionally and systemically confused and teased and tempered until any concept of reality, reason or inherent instinct is numbed, dumbed and strummed.

their myopic greed and selfish lust for power and superiority is killing america fast. -- the empire is destined to crumble. -- it's a party baby: we'll take everything we can and let the next generation(s) clean up the mess. just the idea that it "can" be fixed in the future diminishes any guilt of conscious . . . a typical human psychological trait. --like a drug addiction.­.. your mind will tell you: it's ok... we'll make up for it later when we come up. --- or like evangelicals: since jesus will be returning to take us all up to heaven real soon why should we worry about the environment? it doesn't matter... we should use up the earth all we want. no reason to worry because salvation will deliver us from the consequences.

so disheartening.

it will be interesting when all us spoiled, entitled americans are faced with recession/­depression massive unemployment, no health care or even basic survival needs . . . millions of american's day job will be standing in soup lines all day just to keep their families alive -- third world status and we're not taking it seriously - - -

but all the war profiteers, insurance, pharma, oil and communications execs and all the politicians who cleared the path, enabled and acted as accessorized these treasonous, greedy bastards will have beautiful beach front island property (many an actual island their own shaped like their favorite continent) next door and down the road from all the other world leaders, executives and global crime figures: dubai.

fuck it.

[DBlake]

John, all is good except one thing, as you definitely know N.H. has a huge number of Independent voters (and even larger this time around) and that is not the case nation wide. So the 18% (undecided until day of primary) should have been anticipated and factored into the margin of error.
.