Before my vacation in August, I used two
to review the sorry state of disclosure with respect to how tightly (or
not-so-tightly) the pollsters screen for likely primary voters, especially
those in early primary or caucus states. Today I want to take a look at another
question we might want to ask about these polls to help sort out the
differences in results among them: What are the demographics?
The issue that becomes much more acute in pre-primary
surveys is that the pollster is trying to determine two things at once: (1)
identify the voters that will participate in the primary and (2) measure the
attitudes and vote preference of those voters. When different primary polls produce
seemingly contradictory results, the culprit is usually the people selected. So
if we want to try to tease out the reasons why polls show different results, we
want to know as much as we can about the "likely voters" they sample.
Just last week, for example, I looked at the differing
results in some polls of Iowa Democratic "likely caucus goers" and found wide variation
in the number reporting past caucus participation. As explained in the post,
that variation appears related to support for various candidates (previous
caucus goers are more likely to support John Edwards, newcomers more apt to
support Hillary Clinton and, to a lesser extent, Barack Obama).
For the Democratic candidates, it is not hard to imagine that
variation in demographic variables like gender, age and race might have similar
effects. For example, various public polls
have shown that Clinton
does better among women and Obama better among younger voters. Clinton and
Obama also dominate among African Americans to the relative detriment of John
Edwards and other candidates. Demographic patterns in polls of Republican primary
voters have been relatively inconsistent, although Giuliani and McCain tend to do better
among moderates and Republican leaning independents.
The demographic composition can vary widely. Consider the African
American percentage of the likely Democratic primary electorate in South Carolina in polls released
over the last several months.
- 56% - Hamilton-Beattie (D) and Ayres-McHenry (R), 4/14-19/2007
- 53% - Garin-Hart-Yang (D), 4/9-12/2007
- 50% - Clemson University Palmetto Poll, 8/20-24/2007
- 43% - Public Policy Polling (D), 5/31/2007
- 40% - Public Policy Polling (D), 8/13/2007
- 40%* - CNN/ORC, 7/16-18/2007 (percentage based on black sample size, may have been unweighted)
- 30% - Chernoff Newman/MarketSearch, 4/9-16/2007
Needless to say, the variation above is huge (although the
relationship between racial composition and the Clinton-Obama result has been
weak so far). While these results certainly cannot all be right, the right
answer is not obvious. African-Americans comprised 47% of Democratic primary
voters in the 2004 South Carolina
primary, according to the network exit
poll, but of course an exit poll is also a survey with potential problems of its
own. And whatever the past result, the composition in 2008 may be different.
The important point is that educated poll consumers will
want to know all they can about the demographics of pre-primary polling, and
unfortunately, such information is very hard to find. I went back to the public
releases from 23 different organizations that have released public polls in the
last six months or so in Iowa, New Hampshire and South
- Hamilton Beattie/Ayres McHenry (SC) - gender age, race, income, party
- PPP (IA, SC) - gender, age, race (SC only), party
- SUSA (NH) - gender, age, ideology
- Time (Iowa) - gender, age, education, race, party affiliation, percentage of past caucus goers
As I recall, the Garin-Hart survey of South Carolina also included some
demographics (as I included their result for race in my post on
5/3), but their PDF release is no longer available online. A few
organizations provide more limited information: The American Research group
routinely provides the percentage of independents included in their samples (but
not standard demographics). The ABC/Washington Post survey of Iowa caucus goers provided results to a
question on past caucus attendance (but nothing more). And two surveys of South Carolina - from Clemson University
and CNN - both provided the African-American composition only.
The national surveys are not much better. Only two
organizations routinely provide full data on the demographic composition of the
subgroups that hear primary trial-heat questions: Cook
Political Report/RT Strategies and Diageo/Hotline.
For more than a month, I have been promising
some ideas about what we might do about this paucity of information. I'll have
more on that in the next post.