08/25/2010 04:41 pm ET Updated Dec 06, 2017

Why Were Polls Off In 2010 Florida Primary Races?

Here's any easy bet to win today in Washington (or anywhere else where true political junkies gather): Where did polling miss the mark most yesterday, in Florida's Republican primary for Governor or Florida's Democratic primary for Senate?

Judging by the tweets I've seen (and my own snap judgment), most of you may be thinking the polls were most off in the Governor's race, where most of the final polls showed Bill McCollum leading. If so, you'd be wrong. The three polls fielded in the last week on the Democratic Senate contest understated Kendrick Meek's margin by an average of 11 percentage points. The three final week polls on the Republican Governor's underestimated Rick Scott's margin by an average of just 5 points (the absolute value of the errors was 7.7; all of these numbers are based on the unofficial count with all precincts reporting).

Thus, we have another example of the pre-election pollster's paradox: The errors that get noticed are those that are just wrong enough to give everyone the wrong impression about the likely winner.

But let's focus on the Republican primary for Governor, for now, since theories are flying about why some polls missed Scott's looming victory. I asked our Pollster.com colleague, University of Wisconsin Professor Charles Franklin, to run one of his patented "bullseye" polling error charts. The chart below displays each poll as a dot, with the vertical axis representing Scott's percentage, the horizontal axis representing McCollum's percentage, and the center of the bullseye representing the actual result.

In this case, two pollsters -- Public Policy Polling (PPP) and Sunshine State News/VSS/Susquehanna Polling and Research -- land in the center ring of the bullseye. Their final surveys yielded the smallest undecided percentages and were also the only two to show Scott ahead.

Pollsters have long debated how to handle the undecided category in measuring poll error. Should we allocate the undecided among the candidates and, if so, how? This chart sidesteps that debate, although keep in mind that polls that get the margin exactly right will fall in the lower left quadrant along an unplotted diagonal line from the center of the bullseye to the lower left corner.

So, aside from getting fewer undecided, why did the PPP and Susquehanna polls get closer to the final result?

One theory, floated by our intern Harry Enten (see the 9:01 p.m. entry in last night's live blog), is that the polls by PPP and Susquehanna used an automated, recorded-voice methodology and drew their random samples from the official list of registered voters. Harry argues that both methods provide a more accurate identification of truly likely voters: The registered voter list because it identifies actual voters (and can make use of their actual history of past voting) and the automated method because voters are theoretically more willing to provide honest answers about their vote intent to a machine rather than a live interviewer.

I have long speculated that automated surveys are better at selecting truly likely voters in especially low turnout races, on the theory that they can identify likely voters more accurately and are better able to interview a narrower slice of the electorate at reasonable cost. In this case, however, PPP's Tom Jensen speculates that their survey was closer to the final result because their they "used a loose screen" in selecting likely voters and thus "picked up more non-typical voters who went for Scott." Yesterday's Republican turnout (1.28 million voters) was slightly larger than the turnout in the August Republican primary for Senate in 2004 (1.16 million) and much larger that the Republican Gubernatorial primary in 2006 (0.96 million).

Another intriguing theory concerns the surprising 10% of the vote received by Mike McCalister, a third Republican candidate for Governor. Politico's Jon Martin observed, via Twitter, that McAlister's showing was perhaps "the biggest surprise of the night" because he aired no television ads, did not appear in the debates and got little media attention. Republican media consultant Mike Murphy suggests, also via Twitter, that some voters may have been confused by the similarity in the names: "McCollum + McCalister = McConfused."

What complicates this issue further is that only one pollster -- Mason Dixon -- offered McCalister as a choice on their vote question, and they showed him getting only 4% to 45% for McCollum. PPP and Quinnipiac University offered only Scott and McCollum as choices (although Quinnipiac recorded 4% who named "someone else" as their choice). So what may have happened is that some Republicans headed to the polls intending to vote for McCollum, but accidentally chose a similar looking name by mistake.

Whatever the explanation, we should always remember that polling in primary elections is more prone to mishaps than polling in general elections. Learning that first lesson, however, is the easy part. Anticipating which polls will be right, and even explaining why they differ after the fact, is much harder.