02/04/2008 09:37 am ET Updated Dec 06, 2017

February 5 Polls: Four Cautions

Over the last 48 hours we have had an avalanche of new polls,** and given the discussion both in our comments section and elsewhere across the blogosphere, everyone seems unsure of what to make of the results and what they say about where things stand, especially in the Democratic presidential race. As is evident from our charts, the trends are highly favorable to both John McCain and Barack Obama, but from there things get murkier, especially in the Democratic race. Here is my sense of what the poll results tell us and what they do not.

The Republican race is easier to gauge, largely because of the "winner-take-all" rules that apply in so many Republican primaries. The National Journal's Campaign Tracker shows that more than two-thirds of the Republican delegates up for grabs tomorrow will be awarded on a winner-take-all basis either by state or congressional district or some combination of the two. As such, John McCain's roughly twenty-point leads in most of the national surveys, combined with similar margins in the winner-take-all-states in the Northeast (New York, New Jersey, Connecticut and Delaware) and narrower leads elsewhere position him to take a commanding delegate lead tomorrow night. Mitt Romney's hopes, on the other hand, ride on surpassing McCain in states like California and Missouri.

The Democratic contest is obviously much closer, although in some ways the process of selecting delegates is more straightforward. The allotment of delegates is proportional to votes in each congressional district and each state. While the rules may make for some odd outcomes in individual states (see more detailed explanations here and here), the allotment across all states should be a good reflection of the overall votes cast. While winning individual states may have symbolic value in terms of the way the media covers the results, the total delegate counts amassed across all states are what really matter.

So what do the polls tell us about how tomorrow's Democratic contest will translate into delegates? While Barack Obama appears to be gaining support, there are four reasons to be cautious about what the various polls are reporting [about where the race will end up -- see the clarification below]:

1) Polls are of little use in the caucus states. Roughly 13% of the Democratic delegates chosen tomorrow are from six states and one territory that hold party caucuses (Alaska, Colorado, Idaho, Kansas, Minnesota, North Dakota and American Samoa). Accurate polling in these contests is next to impossible because past turnout has been so light. Fewer than one percent of the eligible adults in the six states participated in the Democratic caucuses in 2004 (ranging from 0.1% in Alaska to 2.2% in North Dakota).

Turnout in the February 5 caucuses is anyone's guess, and as such, pollsters have wisely stayed away. We have logged only two polls in the six caucus states fielded since December. One of these was the Minnesota Public Radio News/Humphrey Institute poll (pdf) that explicitly warned it was "not a prediction of Tuesday night's precinct caucuses" because "the interviews did not identify likely caucus participants." The second, a Mason-Dixon survey in Colorado, is now nearly two weeks old and gave no indication what percentage of Colorado adults were deemed "likely caucus goers."

2) National polls may be misleading. Given the proportional allotment of delegates across such a large number of states, the national polls may provide a reasonable assessment of where the race stands. While we have a lot of very recent national polling data showing Barack Obama gaining, we have to remember that the February 5 states may look different than those not holding contests tomorrow.

So far, I have seen only two national surveys attempt to break out results for the February 5 states, and those show contradictory results.

The report released yesterday by the Pew Research Center allows a comparison across their last three surveys of Democrats in the February 5 states to those who will vote in later primaries. In the December and January surveys, Pew showed no significant difference between these two categories of states. Now, however, Obama does slightly (though not quite significantly) better in the February 5 states. Looking at it another way, virtually all of recent Obama's gains on the Pew survey have come from the February 5 states.

02-04 Pew-2-5.png

On the other hand, the new CBS News survey, which shows the national Clinton-Obama contest deadlocked at 41% each for Clinton and Obama, yields the opposite result. The CBS summary reports the following about a similarly small sample of Democratic primary voters:

The picture in the states voting on Super Tuesday is not nearly as close as the overall picture and offers some good news for Clinton. Among voters in those states, she leads Obama, 49 percent to 31 percent, with 16 percent still undecided.

As Josh Marshall points out, the entire CBS survey was based on 491 Democratic primary voters, so the subgroup of February 5 state voters may have been as small as 200 interviews.

Perhaps our friends at Gallup, who have interviewed nearly 2,200 Democrats over the last five days, can run a tabulation that helps clarify how the February 5 states compare to the rest of the nation. [Update: They did just that -- details here].

3) Are they sampling truly "likely voters?" Some national surveys, such as ABC/Washington Post and CBS, have reported the results of respondents who describe themselves as likely primary voters. Others, however, have reported on the views of registered voters or adults that identify as Democrats. While turnout is likely to be higher tomorrow than in 2004, the percentage of adults that vote in the Democratic primaries is still likely to be smaller than the percentage represented by most of these national surveys.

Here are two sets of turnout statistics to chew over. First, consider how turnout has increased in the Democratic contests held so far:

02-04 turnout so far.png

As should be obvious, turnout has increased dramatically in all the early states even (or perhaps especially) in states that featured little or no active campaigning by the candidates. If nothing else, this pattern suggests that turnout will exceed 2004 levels in all the February 5 states.

It is still worth considering that past turnout has amounted to a relatively small percentage of eligible adults in each state. The following table shows the turnout levels from 2004 for the February 5 primary states as a percentage of all adults an of eligible adults (as reported by Michael McDonald):

02-04 turnout 2004.png

Here's the main point: Even if Democratic turnout doubles tomorrow as compared to 2004, the percentage of adults participating in the Democratic primaries will still be a fraction of the adults identified as Democrats or Democratic "primary voters" on most national polls. Do the truly "likely" voters look different than all Democratic identifiers? Are the statewide surveys doing a better job of selecting "likely voters" than the national polls? Unfortunately, we can only guess, as only a small handful of the statewide surveys report the percentage of adults that their likely voter samples represent.

4) Uncertainty remains high. If you pay attention to nothing else, remember this: As in New Hampshire, a lot of Democrats are having a hard time deciding between Hillary Clinton and Barack Obama. According to the Pew Research Center, both candidates now receive overwhelmingly positive ratings from Democrats:

  • Clinton: 80% favorable, 15% unfavorable
  • Obama: 76% favorable, 16% unfavorable

Again, as in New Hampshire, voters are expressing considerable uncertainty. In California, for example, both the Mason-Dixon and Rasmussen surveys report 29% of Democrats as either completely undecided or indicating there is still a chance they could change their mind about their preference.

This high degree of uncertainty creates the potential for a volatility that the final tracking polls may not reveal. Many voters will likely carry their sense of indecision into the voting booth, so the news and events of the next 24 hours could prove crucial.

Update: Adam's question in the comments suggests the need for a clarification. I have no doubt that support for Barack Obama has been increasing steadily over the last week. Virtually all of the surveys in all of the states are showing evidence of that trend, and as each pollster measures the same population (however it is defined), those trends are reliable. What I am urging caution about is where Clinton-Obama contest ends up when votes are cast tomorrow. As my AAPOR colleague, Professor Robert Shapiro put it over the weekend, "I would trust the trends but not the magnitude - [it] could be greater or less."

**If you have appreciated the constant flow of updates over the weekend, please post a thank you to the indefatigable Eric Dienstfrey for his exceptionally hard work (and for putting up with a boss who sometimes misspells his name).

Correction: The original version of this post incorrectly identified the CBS News survey as a CBS/New York Times survey.