In response to the dialogue we've been having about the Gallup Daily tracking survey (here and here), Gallup's editor-in-chief Frank Newport sent the following response. Say what you will about Gallup, they are consistently among the most transparent and responsive of the public pollsters.
We are always glad to discuss and analyze Gallup poll data. We generally learn from the insights, comments and questions of others.
The particular reader to whom Mark spends time responding was focusing on the fact that Gallup's daily election tracking was not in exact sync with the vote totals across the 22 Super Tuesday states.
We never reported the Daily Tracking results as projective of what would happen on Super Tuesday. Had that been our intention, we would have used a strict likely voter screen. We would have made specific assumptions about what turnout would be in each state and adjusted each state accordingly. This is what we normally do when trying to predict the actual vote in a state or national election. We did not design the tracking survey methods for that purpose. The general patterns of trends among the broad sample of voters we look are extremely important. But the exact numbers are not projections of the vote in any state or combination of states.
As we reported, candidate support levels in the Super Tuesday states were not dramatically different from the national support levels. This suggests that the momentum and trends observed nationally could be hypothesized to be reflected in the Super Tuesday states.
But for a reader to take that as a prediction by Gallup about the precise vote outcome in all Super Tuesday states (or certainly any individual state) is incorrect.
Our data suggested that among all voters across the country and in Super Tuesday states prior to Feb. 5th, Hillary Clinton had a lead over Barack Obama. Of course not all voters went to the polls -- they never do. Initial estimate are that there was only an average 30% turnout - and a turnout which varied widely across states.
The Gallup Daily election tracking uses a mild screen that filters out just those respondents who say they are not likely to vote in response to a four part question. For Republican voters in February so far that has been 16.9%. For Democratic voters it has been 13.7%. In other words, the screen leaves in more than 80% of national adults, making it functionally similar to the typical registered voter screen.
It certainly wouldn't be expected that a large sample of 80% + of all adults would mirror the actual vote total in a widely disparate group of states with on average just about 30% turnout - and with different turnout within each state. By way of example, when we retrospectively go back and look at the sample of voters from Super Tuesday States from the last five days before Super Tuesday -- screened only among those who are extremely likely to vote -- we find that the vote totals are near a tie, with Obama at 48% and Clinton at 45%.
But we didn't get into that before Super Tuesday because that was not our purpose. The purpose of the national tracking is to monitor the mood of all Democratic and all Republican voters across the country as this primary season progresses. After Jan 3rd, of course, some of these people had already voted, and that proportion continues to go up.
One of the great values of Gallup's tracking is the ability to monitor on a daily basis the changing dynamics of the campaign and to see where the momentum is. (The second value is to be able to aggregate data and look at detailed subgroup analysis). Obama had been gaining in the week or two prior to Super Tuesday to the point where he was essentially tied with Clinton among the broad sample of all voters. But then Clinton retook the momentum. Thus, we hypothesize that had the election been held on Saturday, for example, it looks like Obama would have done better than he eventually ended up doing. But we were not attempting to say what the exact vote totals would be.
[UPDATE (2/10)]: The comments left for this this entry are unusually well-expressed and definitely worth a read. They have inspired a few additional thoughts of my own (delayed, admittedly, by a much needed 36 hour break):
First, we ought not pick just on Gallup. Gallup's broad approach to selecting the "voters" that get asked presidential primary questions is more or less what the other national polls do. I first wrote about this issue almost a year ago and warned about it just last week, on the eve of Super Tuesday when headlines told us of a "dramatic shift" toward Obama.
Second, I am certainly sympathetic to the nearly insurmountable challenges that would be involved in creating a combination actual (past) voter/"likely voter"/"likely caucus goer" model that would apply at the national level and somehow take into account the myriad of different rules for participation and historically varying turnout rates. It would not be at all easy.
Also, be careful what you wish for: Those who remember Gallup's daily during the 2000 election will recall that they applied their "likely voter model" to data as early as Labor Day. Critics made a strong case that while the model works well a week before the election it introduces a lot of variation in the kinds of voters selected as "likely," much of it questionable.
Third, I agree with Mark Lindeman that there is value to Gallup's approach. "it's very interesting," he wrote, "to know what Democrats and Republicans (including leaners) around the country are thinking of "their" candidates, whether their states have already voted or not." However, I tend to agree even more with reader DTM's reaction:
[Quoting Newport] "One of the great values of Gallup's tracking is the ability to monitor on a daily basis the changing dynamics of the campaign and to see where the momentum is."
I think it is fair to say the campaigns are directed at eventually getting actual votes in caucuses and primaries, and the kind of momentum the campaigns care about is the kind of momentum that would further such an end. But given the way in which Gallup is defining "voters", the relationship between what is going on in their tracking polls and what the campaigns are actually trying to accomplish is less than clear.
And this is precisely the sort of confusion which worries me. Indeed, they seem to be more or less encouraging people to use these tracking polls for "horse race" coverage, while at the same time admitting they are not really even trying to screen for actual voters in the upcoming contests, which is what the "race" is all about.
Most people who follow the national poll numbers -- including journalists and political professionals -- treat them as if they measure the views of actual voters in party primaries or caucuses. Pollsters could do a much better job making it clear that they also include far more "leaned partisans" than are likely to actually participate in the party primaries and caucuses (regardless of what respondents claim on vote likelihood questions).