Following up on yesterday's
post, in which I speculated - wrongly, as it turns out -- about the incidence
of eligible adults selected by the American Research Group (ARG) as likely
caucus goers for their most recent surveys of Democrats
and Republicans
in Iowa. I
emailed Dick Bennett, and can now report on how their surveys compare to the
others that have provided us with similar details.
First, according to Bennett, I was incorrect in speculating
that they use only one question to screen for "likely caucus goers." They start
with a random digit dial (RDD) sample of adults in Iowa
in households with a working telephone and then ask four different questions (although
they provide only the last question on the page
reporting Iowa
results):
- Theyask whether respondents are registered to vote, and whether they areregistered as Democrats or Republicans. Non-registrants are terminated andnot interviewed.
- Theyask registrants how likely they are to participate in the Caucus "a1-to-10 scale with 1 meaning definitely not participating and 10 meaningdefinitely participating." Those who answer 1 through 6 are terminated andnot interviewed.
- Theyask unaffiliated registrants ("independents" registered as neitherDemocrats nor Republicans) whether they plan to participate in theDemocratic or Republican caucus. Registered Democrats and independents whoplan to caucus with the Democrats get the Democratic vote question;Registered Republicans and independents who plan to caucus with theRepublicans answer the Republican question.
- After askingvote question, they asks the question that appears on the web site: "Wouldyou say that you definitely plan to participate in the 2008 Democraticpresidential caucus, that you might participate in the 2008 Democraticpresidential caucus, or that you will probably not participate in the 2008Democratic presidential caucus?" Only the definite are included in the finalsample of likely caucus voters.
So the process involves calling a random sample of adults
until they reach a quota of 600 interviews for voters of one of the parties. In
their most recent Iowa
survey, they were able to fill the quota for Democrats first, so they continued
dialing the random sample until they had interviewed 600 Republicans,
terminating 155 Democrats in the process. Bennett reports that they also terminated
another 4,842 adults on their various screen questions (740 who say they were
not registered to vote, 3,598 who rated their likelihood of participating as 6
or lower and 504 who were less than "definite" about participating on the final
question).
So, the "back of the envelope" calculation for ARG is that
their most recent sample of Democrats represents 12% of Iowa adults (755 Democrats divided by 755+600+4,842).
Their most recent sample of Republicans represents roughly 10% of Iowa adults (600 Republicans
divided by 755+600+4,842). We can compare the Democratic statistic to those provided
by other Iowa
pollsters:
- 12% -ARG (August)
- 12% - Time/SRBI (August)
- 12% - ABC/Washington Post (August)
- 29% - Universityof Iowa (August)
- 9% - DesMoines Register/Selzer & Co. (June 2006)
And again, for those just joining this discussion, the 2004
Democratic caucus turnout was reported as 122,200, which represented 5.4% of
the voting age population and 5.6 of eligible adults.
So, if we take all of these pollsters at their word, my "blogger
speculation" yesterday was off-base: ARG's incidence of Democratic likely
voters as a percentage of eligible adults is very close to the surveys done by Time and ABC/Washington Post. Apologies to Bennett.
But we still have a mystery. Why the consistent difference
between the result from ARG and other surveys that appears to favor Clinton? Professor
Franklin is working on a post as I speak that will chart the difference, but
when we exclude the ARG's surveys from our estimate for Iowa, Clinton's current
2 point margin over Edwards (26.2% to 24.2%) becomes a 1.3 point deficit (24.6%
to 25.9%). [See Franklin's in-depth discussion, now posted here].
I asked Bennett whether he had any theories that might
explain the difference. Here is his response:
Our sample size is larger and our likely voter screen
is more difficult to pass. As you have pointed out, many surveys (although
they are not designed to project participation) project unrealistic levels of
participation. A likely voter/participant does not need to vote/participate to
represent the pool of likely voters/participants, but the likely
voter/participant pool is not much larger than the actual turnout.Our results in Iowa
show that John Edwards has a slight lead over Hillary Clinton among those
voters saying they have attended a caucus in the past. Hillary Clinton has a
greater lead among those saying this will be their first caucus. Hillary
Clinton also has very strong support among women who say they usually do not
vote/participate in primary/caucus races - this is true in Iowa and the other early states
Sample size is largely irrelevant to the pattern in our
chart. Smaller samples would explain greater variability, but not a consistent
difference across a large number of samples. The observation in his second
paragraph is much more important. Since ARG's previous releases did not mention
these results, I asked for the question about past caucus participation and the
associated results. His response:
The question is: Will this be the first Democratic caucus
you have attended, or have you attended a Democratic caucus in the past?We first asked this in Feb:
Feb - 41% first, 59% past
Mar - 44% first, 55% past
Apr - 39% first, 60% past
May - 45% first, 55% past
Jun - 42% first, 57% past
Jul - 40% first, 60% past
Aug - 43% first, 57% past
We can compare this result to similar questions or reports
from other recent surveys and they show a clear pattern. The differences among
the four pollsters are huge and show a clear pattern, consistent with the
differences Bennett reports in his own surveys: John Edwards does better against Clinton
as the percentage of past caucus goers increases.
So what is the right number of past caucus goers? Bennett
can certainly argue that the entrance polls from the 2000 and 2004 Caucuses are
on his side. Bennett used exactly the same question as the network entrance
poll, which reported the percentage of first-time Democratic caucus goers as 53%
in 2004
and 47% in 2000. Of course, as we learned three years ago, exit polls have their
own problems, and I am guessing that other pollsters will debate what past-caucus goer number is correct. We will
pursue this point further.
Finally, it is worth saying that this exchange and my
arguably unfair "blogger speculation" yesterday makes one thing clear: If we
are going to dig deeper into these issues, we have an obligation to ask these
questions (about incidence and sample characteristics) about all polls, not
just those from ARG, Time and a
handful of others.
Stay tuned.