Earlier today, I participated in a blogger conference call with Gallup's Frank Newport, in which he talked about their latest surveys in California and New York, as well as their intriguing new project, the Gallup Daily. Starting in early January, Gallup started completing 1,000 interviews each night. Starting last week, they are now reporting a three-day rolling average of the presidential primary trial-heat questions. They will release new results each day at approximately 2:00 p.m eastern time -- today's results are here, each new release can be found here along with a daily report on consumer confidence.
True polling junkies will know about another similarly structured poll tracking program: Rasmussen Reports has put out daily updates on the President Bush's job approval rating for many years, and has been reporting three-day rolling average presidential primary tracking since October. Of course, the Rasmussen data is collected with an automated methodology known as interactive voice response (IVR), in which respondents hear a recorded message and answer using the keys on their touch-tone phones.
Whatever you think of IVR polls -- and the methodology certainly remains controversial in the survey research world -- the Gallup program is distinct from Rasmussen in other ways than the use of live interviewers. According to Newport, the Gallup Daily uses the "same robust methodology" as all other Gallup polls: live interviewers, a random-digit-dial sample, as many as 5 "call-backs" to those not home when they call, cell-phone sampling to reach those in cell-phone only households (something Gallup also introduced to their standard methodology this month) and Spanish speaking interviewers available for when they reach a household in which only Spanish is spoken.
That "call back" procedure may sound excruciatingly wonky, but it is important and a key distinction from the Rasmussen tracking. So far at least, Gallup has used a procedure that dialed each sampled number as many as five times over successive nights if the initial attempts were unsuccessful (that is, if the number was busy or if no one answered the telephone). They structure their calling procedure so that the sample on any given day is equivalent to a sample dialed for many days. Each day has the same mix of attempts (first, second, third and so on).
The Gallup Daily survey design has two critical benefits, according to Newport. First, obviously, it allows us to attempt to monitor the impact of major campaign events on a daily basis, such as the Barack Obama's victory in South Carolina. Newport reported that their sample last night had Clinton ahead by 10 percentage points (he did not provide specific percentages), while the result over the last three nights had Clinton leading Obama by 11 points (43% to 32%). So Newport's sense is that the South Carolina results has not yet dramatically changed the Democratic standings.
However, a quick glance at their chart shows something else that Newport confirmed. The Democratic race has narrowed significantly over the last week. Clinton lead by 20 points (48% to 28%) on their January 20 release but, again, by only 11 points today (43% to 32%). Given the large samples involved (approximately 1,200 leaned Democrats for each release), both the Clinton decline and Obama increase over the last week are statistically significant.
The second "tremendous virtue," as Newport put it, is the ability to pool very large data sets for subgroup analysis. Consider that Gallup will be interviewing 7,000 adults each week and over 30,000 adults a month. As such, they can pool data and examine demographic subgroups to a degree impossible with once-every-two-week samples of 1,000. As an example of this "unique and powerful ability," Newport cited his recent analysis of the Democratic primary contest within socio-economic differences of African Americans. Ordinarily, in a sample of 1,000 adults, the total number of black Democrats would be well under 100 respondents. But with over 20,000 interviews to work with, Newport was able to compare the presidential preference of African-Americans earning less than $24,000 a year to those making $90,000 or more. That's extraordinary.
This program has one other virtue that Newport did not mention, and that involves the vast analytical resources at Gallup. Given their ability to do something with all that data, it is the sheer number of interviews that will make the contribution to our understanding of politics and the presidential campaign during 2008, especially given Gallup's commitment to methodological transparency and disclosure. Hat's off to Frank Newport and Gallup. This is truly a big deal.