WASHINGTON -- A day or two after an election, those who follow politics closely like to ask, "Who was the best pollster?"

That's a great question, but a tough one to answer even under ideal circumstances. Trying to answer it right now, with millions of votes still uncounted, is misleading at best.

On Tuesday, however, Costas Panagopoulos, director of Fordham University's Center for Electoral Politics, published just such a ranking of 28 organizations that had produced a national survey in the closing weeks of the campaign. Not surprisingly, news sites like TPM and The Washington Post quickly rushed out reports that reproduced the list and focused on the pollsters at the top.

Here are three reasons why this sort of ranking falls flat.

First, the Panagopoulos rankings (which we haven't reproduced here) are based on just one national survey from each organization (or combination of pollster and sponsor). Together, those surveys produced a relatively narrow range of results (see below). For example, 14 of the 28 polls gave President Barack Obama nominal leads of 1 to 3 percentage points; 23 of the 28 ranged between a tie and a four-point Obama advantage.

2012-11-08-final281.png



Second, the calculation makes no allowance for the margin of sampling error -- that is, the range of expected random variation that for most of the surveys was at least plus or minus 3 percent on the estimate of support for each candidate. As a result, many of the polls on this list could have found a margin of 1 or 2 or 3 points favoring Obama purely as a matter of chance.

Yet a percentage point or two is all that separates the "best" pollsters on the list from those who rank in the middle. As of this writing, Obama leads Mitt Romney in the popular vote count by 2.4 points (50.4 percent to 48.0 percent). Not surprisingly, the polls that showed Obama winning by a 2-point margin -- from Public Policy Polling, Ipsos and YouGov -- ranked at the top of the Panagopoulos list (he released no specific scores for each pollster or estimates of the precision of the scoring).

Third and most important, the votes are still being counted. As of this writing, there are still millions of uncounted provisional and absentee ballots in Ohio, Florida, Arizona, California and many other states.

On the Thursday after the 2008 election, just over 121 million votes had been counted, giving Obama a 6-point lead over Sen. John McCain (52.3 percent to 46.3 percent). But those totals did not include nearly 10 million additional votes that were later included in the official certified count, boosting Obama's margin over McCain to 7.4 points (52.9 percent to 45.5 percent).

If Obama's margin over Romney shows a similar increase this year, it could easily rise to 3 or even 4 percentage points, which will cause the pollsters that Panagopoulos currently ranks as "best" to fall to the middle and other organizations to rise to the top.

That's exactly what happened four years ago. Panagopoulos rushed out a ranking on the day after the election and then revised and republished it later when the count was complete. The result? The six "best" pollsters in his initial ranking were displaced by six entirely different "best" pollsters a few months later.

If we want to assess how the national pollsters did on the basis of their final surveys alone, a far better approach is to determine whether those polls did what they were supposed to do, which was to capture the actual result within their expected margin of error. That sort of comparison can be seen in the chart below (created by George Washington University political scientist John Sides).

2012-11-08-yougovaccuracy.png



The chart shows not just how far each final poll's estimate varied from Obama's current percentage of the two-party vote (ballots cast for either him or Romney), but also the range of each poll's margin of error. According to the chart, nearly every poll performed as it should have, which is to say Obama's current percentage falls within its margin of error. (Remember, however, that the vertical line representing Obama's actual vote will likely move slightly to the left once all the ballots are counted.)

Scoring pollsters on the basis of their final poll has other problems as well. Some of the surveys on the Panagopoulos list, such as the Associated Press/GfK survey, were conducted more than a week before the election. As the HuffPost Pollster trend chart shows, those polls likely missed a final-week uptick for Obama captured by most of the daily tracking surveys.

Moreover, polling firms can sometimes be influenced by the results reported by other polling firms; they may herd together, especially as an election ticks down. That means the very last survey may not be the best indicator of the overall quality of a pollster's work.

Given all the issues, naming a "best pollster" in this way is not a good idea. But it's not at all premature to begin to assess the performance of pollsters generally or to identify the firms whose results were especially problematic.

For example, while the averages compiled by HuffPost Pollster and the other polling aggregators were correct in forecasting Obama the winner of the key swing states, HuffPost's averages understated the president's victory margin by 2 to 3 percentage points in Wisconsin, Nevada, Iowa, New Hampshire and Colorado (as of this writing, based on the current AP vote count).

2012-11-08-battlegroundaverages.png



That pattern may be connected to the consistent house effects demonstrated by certain pollsters throughout the fall campaign. The chart below was compiled by Simon Jackman on Oct. 24, two weeks before the election, but the relative ranking of the pollsters showing the most pronounced house effects did not change significantly over the last two weeks.

2012-10-24-deltaForHuffPost.png



Pollsters like Rasmussen Reports, Gravis Research and Gallup produced results that were consistently more favorable to Romney throughout the fall. While additional analysis is necessary, the big house effects from these especially prolific pollsters help to explain the understatement of Obama's vote by polling aggregators.

Crowning a single "best pollster" now on the basis of just one poll from each organization is a futile and misleading effort. For the moment, assessments of pollster error should focus instead on those that missed the mark by wide margins.

CORRECTION: An earlier version of this story credited the margin-of-error chart to YouGov. In fact, it was created by George Washington University political scientist John Sides and later tweeted about by YouGov.

Also on HuffPost:

Loading Slideshow...
  • President Barack Obama and wife Michelle hold hands with Vice President Joe Biden and his wife Jill following Obama's victory speech to supporters in Chicago early Wednesday, Nov. 7 2012. (AP Photo/Jerome Delay)

  • Barack Obama

    Jill Biden watches as Vice President Joe Biden is hugged by first lady Michelle Obama at President Barack Obama's election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Nam Y. Huh)

  • Barack Obama, Michelle Obama, Sasha Obama, Malia Obama

    President Barack Obama, accompanied by First Lady Michelle Obama and daughters Malia and Sasha arrive at the election night party Wednesday, Nov. 7, 2012, in Chicago. Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Carolyn Kaster)

  • Barack Obama

    President Barack Obama looks at his daughter Sasha as he walks with his wife Michelle and daughter Malia at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/M. Spencer Green)

  • Barack Obama, Michelle Obama, Sasha Obama, Malia Obama

    President Barack Obama, accompanied by First Lady Michelle Obama and daughters Malia and Sasha arrive at the election night party Wednesday, Nov. 7, 2012, in Chicago. Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Carolyn Kaster)

  • Confetti covers the stage after President Barack Obama speaks at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/=2121300183=)

  • Barack Obama, Michelle Obama, Jill Biden Joe Biden

    President Barack Obama with First Lady Michelle Obama, Vice President Joe Biden and Jill Biden celebrate on stage at the election night party at McCormick Place, Wednesday, Nov. 7, 2012, in Chicago. Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney.(AP Photo/Carolyn Kaster)

  • US-VOTE-2012-ELECTION-OBAMA

    US President Barack Obama celebrates with First Lady Michelle Obama on stage on election night in Chicago on November 6, 2012. Obama swept to re-election, forging history again by transcending a slow economic recovery and the high unemployment which haunted his first term to beat Republican Mitt Romney. AFP PHOTO/Jewel Samad (Photo credit should read JEWEL SAMAD/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    US President Barack Obama celebrates after delivering his acceptance speech ion Chicago on November 7, 2012. Obama swept to re-election, forging history again by transcending a slow economic recovery and the high unemployment which haunted his first term to beat Republican Mitt Romney. AFP PHOTO/Jewel SAMAD (Photo credit should read JEWEL SAMAD/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    Confetti rains on the stage as US President Barack Obama celebrates his victory in the presidential election in Chicago on November 7, 2012. Obama swept to re-election, forging history again by transcending a slow economic recovery and the high unemployment which haunted his first term to beat Republican Mitt Romney. AFP PHOTO/Jim WATSON (Photo credit should read JIM WATSON/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    US President Barack Obama and First Lady Michelle wave to supporters following his election victory in Chicago, Illinois on November 6, 2012. Obama swept to re-election, forging history again by transcending a slow economic recovery and the high unemployment which haunted his first term to beat Republican Mitt Romney. AFP PHOTO/Jewel Samad (Photo credit should read JEWEL SAMAD/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    US First Lady Michelle Obama hugs President Barack Obama on stage following his election victory in Chicago, Illinois on November 6, 2012. Obama swept to re-election, forging history again by transcending a slow economic recovery and the high unemployment which haunted his first term to beat Republican Mitt Romney. AFP PHOTO/Jewel Samad (Photo credit should read JEWEL SAMAD/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    US President Barack Obama waves to supporters following his victory speech on election night in Chicago, Illinois on November 6, 2012. Obama swept to re-election, forging history again by transcending a slow economic recovery and the high unemployment which haunted his first term to beat Republican Mitt Romney. AFP PHOTO/Jewel Samad (Photo credit should read JEWEL SAMAD/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    US President Barack Obama smiles as First Lady Michelle Obama gives the thumbs-up following Obama's speech on election night November 6, 2012 in Chicago, Illinois. President Barack Obama swept to re-election Tuesday, forging history again by transcending a slow economic recovery and the high unemployment which haunted his first term to beat Republican Mitt Romney. AFP PHOTO/Jewel Samad (Photo credit should read JEWEL SAMAD/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    (L-R) US First Lady Michelle Obama, US President Barack Obama, US Vice President Joe Biden and his wife Jill Biden celebrate on election night November 7, 2012 in Chicago, Illinois. Obama and Biden won re-election to a second 4-year term. AFP PHOTO / Saul LOEB (Photo credit should read SAUL LOEB/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    (L-R) US First Lady Michelle Obama, US President Barack Obama, US Vice President Joe Biden and his wife Jill Biden celebrate on election night November 7, 2012 in Chicago, Illinois. Obama and Biden won re-election to a second 4-year term. AFP PHOTO / Saul LOEB (Photo credit should read SAUL LOEB/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    US President Barack Obama (2nd L), First Lady Michelle Obama (L), Vice-President Joe Biden and Second Lady Jill Biden wave to supporters following Obama's speech on election night November 6, 2012 in Chicago, Illinois. President Barack Obama swept to re-election Tuesday, forging history again by transcending a slow economic recovery and the high unemployment which haunted his first term to beat Republican Mitt Romney. AFP PHOTO/Jewel Samad (Photo credit should read JEWEL SAMAD/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    US President Barack Obama waves to supporters on election night November 7, 2012 in Chicago, Illinois. Republican Presidential candidate Mitt Romney has conceded the race to Obama. AFP PHOTO / Robyn Beck (Photo credit should read ROBYN BECK/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    US President Barack Obama flanked by First Lady Michelle Obama (L) and Vice-President Joe Biden and Second Lady Jill Biden wave to supporters following Obama's speech on election night November 6, 2012 in Chicago, Illinois. President Barack Obama swept to re-election Tuesday, forging history again by transcending a slow economic recovery and the high unemployment which haunted his first term to beat Republican Mitt Romney. AFP PHOTO/Jewel Samad (Photo credit should read JEWEL SAMAD/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    US President Barack Obama (2nd L), First Lady Michelle Obama (L), Vice-President Joe Biden and Second Lady Jill Biden wave to supporters following Obama's speech on election night November 6, 2012 in Chicago, Illinois. President Barack Obama swept to re-election Tuesday, forging history again by transcending a slow economic recovery and the high unemployment which haunted his first term to beat Republican Mitt Romney. AFP PHOTO/Jewel Samad (Photo credit should read JEWEL SAMAD/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    US President Barack Obama waves to supporters beside US First Lady Michelle Obama after winning the 2012 US presidential election November 7, 2012 in Chicago, Illinois. Obama swept to a emphatic re-election win over Mitt Romney, forging new history by transcending a dragging economy and the stifling unemployment which haunted his first term. AFP PHOTO / Robyn Beck (Photo credit should read ROBYN BECK/AFP/Getty Images)

  • US-VOTE-2012-ELECTION-OBAMA

    US President Barack Obama (2nd L), First Lady Michelle Obama (L), Vice-President Joe Biden and Second Lady Jill Biden wave to supporters following Obama's speech on election night November 6, 2012 in Chicago, Illinois. President Barack Obama swept to re-election Tuesday, forging history again by transcending a slow economic recovery and the high unemployment which haunted his first term to beat Republican Mitt Romney. AFP PHOTO/Jewel Samad (Photo credit should read JEWEL SAMAD/AFP/Getty Images)

  • Barack Obama

    President Barack Obama hugs his family at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Charles Rex Arbogast)

  • Barack Obama

    President Barack Obama acknowledges the crowd at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/M. Spencer Green)

  • Barack Obama

    President Barack Obama speaks at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Pablo Martinez Monsivais)

  • Barack Obama

    President Barack Obama kisses his wife Michelle as he walks out to the stage at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Pablo Martinez Monsivais)

  • Barack Obama

    President Barack Obama addresses the crowd at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/M. Spencer Green)

  • Barack Obama

    President Barack Obama kisses his wife Michelle as he walks out to the stage with his daughters Malia and Sasha at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Pablo Martinez Monsivais)

  • Barack Obama, Michelle Obama, Sasha Obama, Malia Obama

    President Barack Obama waves as he walks on stage with first lady Michelle Obama and daughters Malia and Sasha at his election night party Wednesday, Nov. 7, 2012, in Chicago. Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Carolyn Kaster)

  • Barack Obama

    President Barack Obama smiles during his speech at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/M. Spencer Green)

  • Michelle Obama

    First lady Michelle Obama walks off the stage with her daughters Malia and Sasha before President Barack Obama speaks at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Pablo Martinez Monsivais)

  • Barack Obama

    President Barack Obama waves to his supporters at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/M. Spencer Green)

  • President Barack Obama speaks at his election night party, Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Jerome Delay)

  • Barack Obama

    President Barack Obama waves at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Nam Y. Huh)

  • Barack Obama

    President Barack Obama speaks at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Nam Y. Huh)

  • Barack Obama

    President Barack Obama acknowledges the crowd at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Nam Y. Huh)

  • Barack Obama

    President Barack Obama waves to his supporters at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/M. Spencer Green)

  • Barack Obama

    President Barack Obama walks out of the stage with his wife Michelle and daughters Sasha and Malia at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/M. Spencer Green)

  • Jon Tester Max Baucus

    President Barack Obama hugs his wife Michelle and daughter Sasha as daughter Malia, left, looks on before speaking at his election night party, Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Jerome Delay)

  • Barack Obama

    President Barack Obama and first lady Michelle Obama wave at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Chris Carlson)

  • Barack Obama

    President Barack Obama, first lady Michelle Obama, Vice President Joe Biden and Jill Biden acknowledge the crowd at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Chris Carlson)

  • Barack Obama

    President Barack Obama, first lady Michelle Obama, Vice President Joe Biden and Jill Biden wave at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Chris Carlson)

  • President Barack Obama, right, applauds his wife Michelle and daughters Sasha and Malia as they leave the stage before speaking at his election night party, Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Jerome Delay)

  • Barack Obama

    President Barack Obama hugs his daugher Sasha after his speech as his wife Michelle and daughter Malia gather on stage at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Pablo Martinez Monsivais)

  • Barack Obama

    President Barack Obama smiles at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Chris Carlson)

  • Barack Obama, Joe Biden

    President Barack Obama hugs Vice President Joe Biden after his speech at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Pablo Martinez Monsivais)

  • Barack Obama, Joe Biden

    President Barack Obama hugs Vice President Joe Biden as his daughter Malia and Sasha look on at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Charles Rex Arbogast)

  • Barack Obama

    President Barack Obama waves to the crowd of supporters at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Charles Rex Arbogast)

  • Joe Biden, Barack Obama

    Vice President Joe Biden greets President Barack Obama's daughters Malia and Sasha after President's speech at his election night party Wednesday, Nov. 7, 2012, in Chicago. President Obama defeated Republican challenger former Massachusetts Gov. Mitt Romney. (AP Photo/Pablo Martinez Monsivais)

  • President Barack Obama waves after delivering his victory speech to supporters gathered in Chicago early Wednesday Nov. 7 2012. (AP Photo/Jerome Delay)

  • Vice President Joe Biden holds on to President Barack Obama after delivering his victory speech to supporters gathered in Chicago early Wednesday Nov. 7 2012. (AP Photo/Jerome Delay)