Forget the circular firing squad among Republicans, now is the time for pollsters and analysts to take a look and figure out who got the statistics game right.
After thousands of surveys, dozens of methodological arguments and a pile of actual, you know, votes to look at, one thing seems clear: most of the pollsters had the election mostly right. (See below for a best/worst breakdown.)That doesn't mean that some of the principal controversies in polling -- sampling of cell phone users, for example -- have been sorted out.
First, on the analyst tip, Nate Silver of fivethirtyeight.com was the big winner. His final projection for Barack Obama's electoral haul, at 349, stood accurate as of Wednesday morning (though North Carolina could yet add to Obama's total). On the popular vote split, Silver was also right on with his prediction of a 52.3-46.3 advantage for Obama.
As for polling firms, the respected Pew firm was right there with Silver, showing a 52-46 national vote breakdown in its final survey. (Though it's important to note that pollsters, unlike analysts, see their principal role as trying to reflect the electorate ahead of election day, as opposed to making predictions.) Rasmussen can also take a bow for getting the national numbers right.
Overall, pollster Anna Greenberg said she was impressed by the extent to which many polling firms were on the money. "I almost wish it wasn't the case, because it would have been more fun," she said. Given the controversies over "likely voter" predictions, how to weight for an expected large African American turnout, and whether cell-phone only users were under-sampled in surveys, however, Greenberg said "pollsters seem to have accurately captured the electorate."
Brent McGoldrick, a pollster who worked on the Hotline/Diageo tracking poll, agreed. "What you're really talking about with questions of the Bradley effect, or cell phones or likely voters, is the composition of electorate. And most polls just got it right. If you looked at the internals of age groups or Hispanics, you see we're even closer to rights. And that's the fundamental question."
Hotline/Diageo played it somewhat safer than other firms by not attempting to project the likely movement of its undecided voters at the end of its tracking poll. The firm showed a 50-45 Obama lead, which seems accurate enough, though it failed to grapple with the break among late-deciding voters. The last NBC-Wall Street Journal poll gave a similar result, 51-43. Meanwhile, the final ABC-Washington Post survey overestimated Obama's level of support slightly, pegging his vote share at 53 percent with some undecideds still to weigh in.
Gallup's much-cited tracking poll was a touch off the mark, by contrast, when it estimated a 55-44 Obama victory in the popular vote on Nov. 3. John Zogby, whose results were a bit volatile last weekend, wound up overestimating Obama's support, too, when he showed a 54-43 lead for the Democrat in his final tracking poll. Other polls that came "close but not quite" in terms of pegging the national vote split included DailyKos/Research2000 and Marist.
Beyond the national vote, pollsters also staked their credibility on the results in battleground states. The results were mixed. Rasmussen predicted Florida and Indiana would go to McCain; neither did. Should North Carolina fall for Obama, that would be the firm's third swing state error.
Zogby fared better on swing states -- correctly projecting nearly all states that were settled on election night (if you forgive his tie in Missouri, which McCain narrowly carried).
"We did well," Zogby told the Huffington Post on Wednesday morning. "We did the right thing weighting for party identification. We did the right thing enhancing the sample for African Americans, Latinos and young people. In the final push, we were off maybe two points on Obama ... and two and half points on McCain. But I do not have a rope in my hand. I am pretty happy."
As for controversies surrounding the growing number of young voters who only use cell phones -- and are thus not reachable on the landlines pollsters depend on to survey voters -- Zogby isn't much concerned, at least for the near future. "We did our own tests, and we knew there was no difference, this year, between cell-phone only young people and landline-accessible young people." In the future, Zogby said "we're going to be more aggressive with interactive, online polling," but for now the state of the profession is secure.
Diageo's McGoldrick agreed, saying: "Every four years we ask the same questions, and there seems to be a temporary resolution before it starts all over again. ... But for the most part we got it right [this year]."
Quinnipiac's latest swing state surveys were also on point: projecting leads for Obama in Ohio, Pennsylvania and Florida. The Ohio Poll also had a bead on its own backyard, predicting an Obama victory.
Overall, as Emory University political science professor Alan Abramowitz often reminded the Huffington Post during the campaign, the polls showed a stable, small lead for Obama for most of the summer and fall, a fact Gallup touched upon in its own post-mortem Wednesday morning. And in the end, that stable, six-point lead in the polls proved an accurate reflection of the electorate.
But, as Greenberg notes, that doesn't mean polling won't undergo more professional turbulence the next time the electorate changes greatly. Noting that young voters didn't turn out in proportionally greater numbers than in 2006 or 2004 (though their preference for Obama was large), she cautioned that the next election will open up some of the same sampling questions. But for now, it appears, most pollsters can give themselves a cautious pat on the back.
BEST AND WORST LAST-MINUTE POLLS:
National:
Best: PEW and Rasmussen, noted above
Worst: Reuters/CSPAN/Zogby, 11/3: Obama 54/McCain 43
Florida:
Best: Public Policy Polling, 11/2: Obama 50/McCain 48
Worst: Fox-Rassumussen, 11/2: McCain 50/Obama 49
Ohio:
Best: CNN/TIME, 10/28: Obama 51/McCain 47
Worst: Strategic Vision, 11/2: McCain 48/Obama 46
Virginia:
Best: Fox-Rasmussen, 11/2: Obama 51/McCain 47
Worst: CNN/TIME, 10/28: Obama 53/McCain 44
Missouri:
Best: Zogby, Rasmussen and others had this race accurately tied at 49/49.
Worst: Politico/InsiderAdvantage, 10/29: McCain 50/Obama 47
Colorado:
Best: American Research Group, 10/30: Obama 52/McCain 45
Worst: YouGov/Polimetrix, 11/1: Obama 55/McCain 40