So in which state did pollsters have a tougher time, New Hampshire or South Carolina? The answer, based on an objective measure of survey error, may surprise you: South Carolina.
SurveyUSA has put together a report card that calculates one measure of poll accuracy for all of the final surveys conducted by every pollster during the presidential primaries so far (there are other measures, but I will leave that subject for another day). Here's their bottom line:
- Average Pollster Error in South Carolina’s Democratic Primary was 16.6 points.
Funny how the psychology of poll numbers works. Two weeks ago, when the final polls showed Barack Obama leading by an average of eight points, but he lost to Clinton by three. Since the winner was a surprise, the election night coverage obsessed over polling problems. Tonight, the polls were off to an even larger degree -- polls showed Obama leading by nine points but he won by 28 -- but since they missed only the magnitude of Obama's margin, the difference goes largely unnoticed.
What happened? In this case, given the differences between live-interview and automated polls, we have decent evidence of what Noam Scheiber termed a "reverse Bradley/Wilder effect," something I wrote about at more length yesterday. Also, at least three of the final polls reported an African American share of the vote in the 42% to 46% range, while the exit polls report that percentage at 55% (though as Mason-Dixon pollster Brad Coker pointed out yesterday, the hard numbers from the South Carolina Secretary of State may look different). So the enormous turnout -- nearly double that of 2004 -- may have thrown off the likely voter models.
The bottom line is that polling is always going to be perilous for primary elections, especially when turnout exceeds all expectations and large numbers of "likely" voters express uncertainty in the final hours before casting their ballots Yet despite the surge in participation, yesterday's Democratic turnout amounted to just 16.5% of South Carolina's eligible adults. Pollsters still had a challenge in terms of selecting likely voters.
The primary season is far from over, and we may be in store for more surprises.