Pollsters Hit And Miss On 2009 Elections

Some unexpected results in Tuesday night's four high-profile elections raise the question of which pollsters got the races right and which got them wrong.

The races that tripped up the polling community the most were the two big ones in New York. Not a single poll had Democrat Bill Owens, who won his congressional race in New York's 23rd district, garnering more than 37 percent of the vote, according to Pollster.com. He ended up with 49 percent.

Siena Poll came close to nailing the support behind the Conservative Party's Doug Hoffman -- putting his backing at 41 percent, four points shy of his actual result. But the same pollster drastically missed Owens' backing -- putting him at 36 percent. Public Policy Polling declared "Hoffman primed for dominant victory," and it projected him winning 51 percent of the electorate, with 34 percent for Owens.

The problem, in the end, was the fluidity of the race. Republican candidate Dede Scozzafava dropped out days before the election and there was a mad dash among statistical scientists to figure out who would get her voters. As PPP wrote in a post-election blog post, its findings were "way off the mark" and "very embarrassing," likely due to the "bizarre machinations in the race..."

The next worst showing among pollsters was in New York City's mayoral race, which turned out to be way closer than projected. Every single poll over the last month showed incumbent Michael Bloomberg with an advantage of at least 11 points. Quinnipiac had 50 percent of the electorate favoring Bloomberg -- which ended up being correct -- but had his challenger, Democrat Bill Thompson, at 38 percent (eight percentage points below where he ended). Marist, meanwhile, had Bloomberg with a 15-point advantage.

In the New Jersey governor's race, some pollsters ended up missing the mark. Pre-election polls showed Republican Chris Christie up by a few points. But the overall average compiled by Pollster.com had the race tied. The end result was a four-percentage point victory (49 percent to 45 percent) for Christie, pulling the rug out from under two firms that had him losing. Monmouth/Gannett and Democracy Corps, projected incumbent Jon Corzine emerging victorious by two and five points respectively.

In the Virginia gubernatorial election, the pollsters called the winner correctly, but slightly understated his margin. The Pollster.com average heading into the election had Democrat Creigh Deeds winning 41 percent of the vote to Republican Bob McDonnell's 54.7 percent. Deeds ended up right at that 41 percent mark. But McDonnell outdid expectations, gathering 58.6 percent of the vote. Survey USA hit the nail on the head when it locked McDonnell in at 58 percent and Deeds at 40 percent in its last survey on November 2.

In the end, of course, polling is an inexact science designed to give viewers a sense of where the electorate stands at a given time and not, necessarily a pitch-perfect prediction of what the result will be. And, like in every other campaign, it will be some time before one can properly assess which pollsters had a strong (or weak) cycle in 2009.

"Yes, there are several good stories about what went right and what went wrong with yesterday's polling, including some important lessons about the value of automated polling," wrote Mark Blumenthal of Pollster.com. "Some pollsters certainly did better yesterday than others. And I'm hoping to have something written and posted on that subject later today, provided that I don't get bogged down by the calls and emails from reporters wanting me to tell them, 'who was the most accurate pollster yesterday?'"

Comments

[FirstShirt]

It's garbage in and garbage out. If you ask Nancy Pelosi and her staff you will get a different perspective than Grayson and his staff. And if you ask a million folks in the inner city that is different than a hundred coming out of an upscale fashion mall.

Still it is fun to watch and get the chills from the possibilities. And, just before an election they are surprisingly accurate.

[billobasher]

The problem with polling is that you can not really know for sure about turnout. Had the New Jersey turnout not been one of their worst ever Corzine would have probably won.

[billobasher]

You mean the "Most accurate" Rasmussen was wrong? He was way off in al three of them.

[jeremyemilio]

Actually the polls were surprisingly accurate... again. It's not a crystal ball, but Pollsters, especially when you combine polls, tend to give a really good idea of who is safe, who is a lost cause, which races are going to be blowouts, and which ones are too close to call before the votes are counted, and especially of trend lines. To cite NY 23 as an example of poor polling is ridiculous. One of the major party candidates dropped out and endorsed the other major party candidate against the insurgent just days before the race. To expect the pollsters to be able to call that race before the dust has settled is lunacy. In a situation like that 24 hours either way and you may have a different outcome altogether. The pollsters knew that it would be relatively close and relatively unpredictable. It was. People who dismiss polls with this "Oh, the polls are always way off" mythology are typically setting themselves up for disappointment.

[Root]

Really, this posting cherry picks statistics to support the story. For example, in the NY-23, there were a significant number of undecideds. So, yes, the pollsters did not get Owen's number exactly right, but some did have him winning, with a substantial number of undecideds unallocated to any candidate. They may have broken towards Owen at the last moment or stayed home; this race was very fluid at the end which wreaked havoc on polling, much like last year's New Hampshire primary. The pollsters who got NJ wrong did not do so terribly, not so far outside normal statistical sampling error that is the margin of error. The only race that had truly questionable polling was the NY mayor race. Here, I suspect that the pollsters did not have a good handle on what constitutes a likely voter in a local election.

[jeremyemilio]

My suspicion is that the pollsters actually influenced the race for NY mayor in an inverse way. People who liked Bloomberg (and there are plenty of them) simply assumed, based on the polls, that he was a shoe in. So they didn't feel any sort of urgency to vote. People who disliked him, on the other hand, while the polling may not have given them much hope of defeating him, were still determined to voice their disapproval through their ballot. Not poor polling, really, but like you say the 'likely voter' thing was simply thrown out of wack and hard to predict. Nevertheless, they were still right in predicting that Bloomberg was safe in the sense that the final result was probably as close to his worst case scenario as possible, and he still got a fairly solid win, expectations aside.

[Philo Beddoe]

How is it that peoples civil rights can be subject to a simple up or down vote?

I dont recall any up or down votes on inter racial marriage bans of the civil rights era, the courts simply ruled them unconstitutional and thew them out.

[jws2346]

Hmmm..., it do add credibility to the ole' sayin' "it ain't over 'till the fat lady sings".

[The Professor]

I would be interested to see how all the major polls stack up against reality rather than just a selected few.

[Fi]

The pollsters like the main stream media, just publish/say whatever they want, they do not let the facts get in the way.
Please America, wake up to your press and demand the truth from them.

[ajwriter]

Post-2008 elections, there is a new factor no one knows how to factor in, and it was the biggest issue in these races: turnout of newly registered voters.

Clearly, there was a bigger turnout of Democratic voters in NY than in Virginia, where only a small, small percentage of those who registered to vote for Obama even turned out for this election. Republicans can always count on a solid turnout by their party faithful, this election was a reminder of that more than anything else.

Democrats better quick figure out how to give their newly minted voters a civics lesson on the importance of turning out for other races, not just Presidential elections.

[Carolab]

"Funny" that these polls were so far off the mark on the Democratic candidates, isn't it? Could it be the pollsters were trying to influence the elections?

[dwright]

that is how I see it exactly

[LMPE]

I never pay attention to polls. I also never watch Fox Noise.

[enquiringmind]

WAR is a racket. It always has been.

It is possibly the oldest, easily the most profitable, surely the most vicious. It is the only one international in scope. It is the only one in which the profits are reckoned in dollars and the losses in lives.

A racket is best described, I believe, as something that is not what it seems to the majority of the people. Only a small "inside" group knows what it is about. It is conducted for the benefit of the very few, at the expense of the very many. Out of war a few people make huge fortunes.

In the World War [I] a mere handful garnered the profits of the conflict. At least 21,000 new millionaires and billionaires were made in the United States during the World War. That many admitted their huge blood gains in their income tax returns. How many other war millionaires falsified their tax returns no one knows.

How many of these war millionaires shouldered a rifle? How many of them dug a trench? How many of them knew what it meant to go hungry in a rat-infested dug-out? How many of them spent sleepless, frightened nights, ducking shells and shrapnel and machine gun bullets? How many of them parried a bayonet thrust of an enemy? How many of them were wounded or killed in battle?



General Smedley Butler, US.Army

[steppenwolfsayshi]

Polls were designed to be used by campaign teams to help them make alterations to their strategy if needed. It's really irresponsible of the media to continuously report them as if they actually indicate how well a candidate is doing when they are so in accurate. Its like the news media can't report anything else. They use the polls to frame the narrative of who is winning and who is losing so they can speculate on the candidates next move, instead of informing the public about the issues of the election so that they know who to vote for.

[Katzencats]

I seem to remember a lot of polling experts saying last year - a year ago today, in fact - something about "The Bradley Effect" in regards to Sen. Obama's chances for winning the Presidential election.

They were wrong then, too. : )

[tel8034]

They were wrong with the women's vote, the hispanic vote, and the youth vote as well.

All the pundits and pollsters had Obama only getting the African American vote and nothing else.