Today, in our ongoing conversation about whether national
surveys may be overstating Hillary Clinton's lead, let's look at some recent
statewide surveys from California.
In that state at least, the data suggest that if anything, the opposite may be
true.
First, consider past turnout in California. On March 2, 2004, 3.1 million
Californians voted in the Democratic presidential primary. Turnout amounted to
roughly 14.8% of eligible adults [and 11.7% of all voting age adults -- see update below].
Now consider four recent surveys, each with a unique
methodology. The Public
Policy Institute of California (PPIC) conducted a survey of 2,000 California adults, March
13-20, using a traditional random-digit-dial (RDD) methodology. They used screen
questions to select, first registered voters, then 498 likely Democratic
primary voters (those who consider themselves "strong Democrats" or say they
plan to vote in the primary on February 5, 2008). The Democratic sample amounts
to 25% of the adults interviewed.
The firm SurveyUSA
polled Californians, March 3-5, using an automated "Interactive Voice Response"
methodology that requires respondents to answer using the keys on their
touch-tone phone, but also uses the traditional RDD methodology to reach a
random sample of all households with a working landline telephone in the state.
They interviewered 2,400 adults and selected 562 likely Democratic primary
voters; the Democrats were 23% of the adult sample.
The Field Poll,
the granddaddy of all California
polls, has recently shifted
to so-called "registration-based-sampling" (RBS). Their March 20-31 survey was
based on a sample drawn from a list of registered voters in California. They interviewed 1,093
registered voters then used screen questions to select 417 562 likely Democratic
primary voters. With a back-of-the-envelope calculation (that California's
15.8 million registered
voters last fall were 74% of the voter eligible
population), the Democrats in the Field poll amount to roughly 28% of California eligible
adults [and 23% of voting age California adults - see Update II below].
Finally, Working
Californians (a labor friendly non-profit) yesterday released a survey of
400 likely Democratic primary voters in California
conducted April 9-12 by Democratic pollster Mark Mellman (who polled for John
Kerry in 2004 but is not currently affiliated with any of the 2008 presidential
candidates). Via email, Mellman confirms that he, like most campaign pollsters,
not only sampled from a voter list but takes the method one step further than
Field. Rather than simply screening for likely primary voters, Mellman also
selected from the registered voter lists only individuals with some past
history of primary voting.
Here are the results:
A few things stand out. First, despite their wide variation
in methodology, all of these surveys use "tighter" screens than most of the
national polls, and yet Hillary Clinton leads by a healthy margin in all four. She
runs as strongly here as in the national surveys, despite the arguably tighter
screens used by PPIC, Field and SurveyUSA.
Second, the one automated poll from SurveyUSA has Clinton
leading by roughly the same margin (13 points) as the two interviewer surveys
conducted at about the same time by PPIC (+11) and Field (+13). SurveyUSA
produced a slightly smaller undecided (6%) than the other polls (9% to 14%), as
they typically do. No evidence of a "don't
tell mama" effect here.
Third, and perhaps more important, the survey with the arguably
tightest screen - the one with the least potential to include non-primary
voters - gives Clinton
her largest lead (+19) and Obama his smallest share of the vote (19%).
UPDATE:Over at MyDD, reflecting in part on my last post on this subject, Chris Bowers has concluded that "the existing evidence no longer provides any clear support to the Inflated Clinton Poll Theory." He adds:
Simply put, there are conflicting indications as to whether Clinton or Obama would benefit more from a tighter poll sample that focused only on Democrats who are likely to make up the caucus and primary electorate, rather than all Democratic self-identifiers and leaners who are registered to vote. Given all of this, I think it is time that I move on and start blogging about other subjects again.
Agreed -- and we will do the same. See you Monday.
UPDATE II: Mark
DiCamillo, director of the Field Poll, left the following comment below:
[W]hen making your "back of
the envelope" calculations to compare The Field Poll's RBS sample to the
other polls' RDD samples, I would take issue with your decision to calculate
registered voters as a percentage of California's citizen-eligible population
(74%) rather than its adult population (66%). When RDD samples are implemented,
the survey universe includes all adults, not just citizen-eligibles. So, if you
had calculated registered voters using its 66% share of all adults, this would
have reduced the proportion of eligible Democrats sampled by The Field Poll's
to 25% of California
adults, rather than 28% of citizen-eligible cited in your post.
He is right, and I apologize for the oversight. The voting
age population (VAP) in California
is a more appropriate statistic because the voter eligible population (VEP) statistic
that I used excludes non-citizens that will be contacted by an RDD survey. As
DiCamillo points out, the adults contacted by an RDD survey will include
non-citizens, especially in California,
a state with the largest percentage of voting age non-citizens in the nation (18.9% in August
2006).
As for his estimate that registered voters are 66% of the
adult population, DiCamillo may have been using older data. GMU Prof. Michael
McDonald's invaluable archive of turnout statistics reports tells us that the
voting age population for California
was 26.6 million in the August 2006, which
would make registered voters 59.5% of that population. As such, the Democrats
on the Field poll represent roughly 23% of California adults.
Of course, the same point applies to my turnout calculation.
The California
Democratic primary turnout of 2004 amounted to 11.7% of that state's adult
population at the
time.
This difference does not affect my larger point: Turnout was
greater for California's Democratic primary in
2004, then for Democrats nationally,
yet California's
pollsters are using tighter turnout screens than the national pollsters. And Clinton's lead still looks
roughly the same as in the national polls.