THE BLOG

A Pollster Grinch Effect?

10/18/2007 07:50 pm ET | Updated May 25, 2011

While pondering some new poll results from Iowa last night, MyDD's Jon Singer asked some good questions:

How do you come up with a turnout model when you
don't know what day the caucuses are going to be held? Specifically, does
anyone actually believe that turnout for a Thursday night January 3 caucus,
when many voters just won't have the time to take two hours to participate,
would be the same as the turnout on a Saturday afternoon January 5 caucus, when
significantly fewer voters will be working or have just gotten off of work?
Might not the turnout also be different were the Democratic caucuses to be held
on Tuesday night January 14, which Ben
Smith
says is a possibility?

It could be the case that the sentiments of voters
1 through 125,000 are not terribly different from those of voters 125,001
through 150,000 or 175,000 or 200,000. But then again, it also could be the
case that those going to caucus for the first time ever or even the first time
in many years are a whole lot different from those who are already pretty
determined to keep up their streak of making it to the caucuses every four
years.

So do we need to consider a "pollster Grinch
Effect
? Does uncertainty surrounding
the date of the Iowa
caucuses make it even more difficult for pollsters to identify and sample "likely
caucus goers?" Yes and yes, but...

While Singer is asking all the right questions, he is probably giving pollsters too much credit for our ability to divine likely caucus goers with laser-like precision, regardless of our assumption about the level of turnout. A public
opinion poll is basically a blunt instrument when it comes to "modeling" likely
caucus participants. The primary measures that most public pollsters use to
select likely caucus goers are self-reports of interest in the caucus, and
intent to participate and (in a few cases) self-reports of past participation. Unfortunately, respondents
notoriously overstate their intent to vote. Most want to show an interest in
doing their civic duty, especially when asked by a stranger on the telephone. So
rather than take responses at face value, most pollsters use several different questions in combination to try to narrow their "likely
voter" subgroup to some reasonable number.

A few public polls in Iowa have sampled from lists of registered voter lists, a procedure that at least provides an accurate way to screen out non-registrants and sort out those registered as Democrats, Republicans
or with no affiliation. But as ABC's Gary Langer points
out
, those lists only eliminate the roughly 17% of the adult population
that is either not registered or identified as "inactive" voters by Iowa's
Secretary of State. The record of actual party affiliation is helpful to
pollsters but not a conclusive indicator of their caucus of choice, since Iowa voters can register or declare their party affiliation on caucus night.

Only one or two public Iowa polls have used actual vote
history to select their respondents, and -- except for the recent polls
conducted for the One Campaign -- none have used past caucus participation to select their likely caucus-goer samples.

So the bottom line is that even if we knew exactly how many voters planned to participate, modeling
the likely caucus goers comes down to methodology decisions that amount to an
educated guess, at best. And even then, we have very little idea how many Iowans
will participate. Consider the estimated turnout from past years (from an
offline source: Rhodes Cook's invaluable Race
for the Presidency: Winning the 2004 Nomination
):

10-18%20iowa%20turnout.png

Look closely at the contested Democratic races, 1980, 1984,
1988, 2000 and 2004. "Estimated" turnout varied enormously, from an estimated 60,000
to 124,000. And as we learned last week,
some have expressed doubt about the 2004 estimate, since caucus organizers ran
out of sign-in sheets and failed to record name and address information for
nearly twenty thousand participants.

And finally, we have to consider that every campaign is
doing everything it can to identify and, ultimately, turn out voters who are
not typical caucus goers. Some are devoting literally millions of dollars to microtargeting,
field staff and various forms of "voter contact" to alter the turnout in their
favor.

So - before we contemplate the Grinch Effect - what level of
turnout is likely in 2008? Who knows?

What this means for the polls we plot and obsess over is
that they are, at best, blunt measures of voter preferences based in Iowa, and no two pollsters
define "likely caucus goers" alike. They do give us a decent sense of trends -
who is gaining or falling -- especially for surveys done by the same pollster
using a constant methodology. However, the "point estimate" for
any candidate on any one poll has a lot of room for error, the kind that has absolutely nothing to do with the
statistical "margin of error."