How Two Polls In The Same State Can Say Two Different Things
Can two polls released on the same day in the same state show two very different results? Yes, and it's happening right now - on the eve of Super Tuesday.
The conundrum was highlighted on Sunday morning on NBC's "Meet The Press" in a discussion of two different California Democratic Primary polls - one conducted by the Field Research Corporation, one from Mason-Dixon Research - that reached two very different conclusions.
Field pegged Clinton and Obama in an almost statistical dead heat: 36 percent for her, 34 percent for him. But Mason-Dixon painted a different picture: Clinton at 45 percent, Obama at 36 - a nine-point gap.
Viewers were left scratching their heads: how is it that two polls in the same state, released on the same day, could reach such vastly different conclusions?
One reason the numbers could have differed so greatly is that Field pollsters asked voters whether they favored "Clinton, Obama or someone else." Mason-Dixon pollsters did not prompt voters for "someone else."
How a company handles undecided voters can make a big difference in the results. Field reported that 12 percent of voters opted for "someone else," while Mason-Dixon pegged that undecided number at 3 percent.
Another explanation is that Californians who vote by mail have been able to cast their ballot for weeks, well before John Edwards and Bill Richardson dropped out of the race
The two surveys were also in the field for different periods of time: Mason-Dixon was conducted over three days, from Jan. 30 to Feb. 1, while Field conducted interviews over eight days from Jan. 25 to Feb. 1. Experts say that the longer the field period, the higher the response rate because they are more likely to reach those people they have identified as likely voters.
Field noted that it makes up to six attempts to reach each voter identified in its sample.
"The longer you can call, the more opportunities you have to reach people at home," said Mark Blumenthal editor and publisher of Pollster.com.
Another possible reason for the two results: the way in which both companies define "likely voter."
Field interviewed 511 likely voters, while Mason-Dixon contacted 400 likely voters. But "likely" doesn't mean the same thing to both companies.
The fact that independents in the Golden State can vote in the Democratic primary, but not the Republican one further complicates determining who is a "likely voter" in California. Field Research does note that that they culled their sample from a list of registered voters.
As to whether voters should believe one poll over another, Mark DiCamillo, director of the Field Poll would only say "I know the methods we use. They are as solid as we can make them."
But Blumenthal stressed that the most important number in this year's election is the number of undecided voters. In both the Field and Mason-Dixon polls, undecideds accounted for 18 and 16 percent respectively.
Blumenthal said that the undecided voters have been the main source of pollster grief this cycle.
As he noted on his own site on Monday:
This high degree of uncertainty creates the potential for a volatility that the final tracking polls may not reveal. Many voters will likely carry their sense of indecision into the voting booth, so the news and events of the next 24 hours could prove crucial.