Update: Cook County Exit Poll

02/04/2010 04:40 pm ET | Updated May 25, 2011
  • Mark Blumenthal Mark Blumenthal is the Head of Election Polling at SurveyMonkey.

Another update, this one on the volunteer exit poll conducted this week in Cook County Illinois by the recently launched Chicago Current. Current editor Geoff Dougherty posted a refreshingly candid postmortem on their efforts:

At 6:11 p.m. yesterday, before the polls closed, I wrote that our exit polling suggested Toni Preckwinkle had the Cook County Board president's race locked down.

And I was right. Our survey honed in on Preckwinkle's strong performance early in the day, and continued to highlight her lead as the election progressed.

And yet ... our poll was wrong. I predicted Preckwinkle would snag 69% of the vote, and noted that the poll had an 8% margin of error. Preckwinkle ended the day with 49% of the vote -- well outside that margin.

Such are the joys and pains of exit polling.

There's more, and it's worth clicking through to read the rest.

I would give the Current an "A" for effort and transparency, but we need to be realistic about the quality of the survey they ultimately produced. Dougherty says it cost just $200, "most of which went for a $100 rental car," and don't think he would argue with the conclusion that they got what they paid for. The poll managed to collect just 93 completed interviews at only 9 of 25 precincts (presumably) selected at random. As Dougherty reported at 1:32 p.m. on Tuesday:

So far we've got about 30 responses. We'll be taking a pause here as our field crew relocates to new spots and starts talking to voters.

We'd originally planned to survey 25 precincts, but logistics are interfering, and we'll probably wind up with about half that. We'd targeted 600 voters, but low turnout will probably leave us with about half of that count.

Never mind the very small sample size. How truly random was the sample? It's hard to tell from this description, but the execution clearly fell short of ideal.

Dougherty says that the "networks often pay tens of thousands of dollars for these things." That's not quite right. I'm not sure how it translates into a per-state cost, but the every-two-year National Election Pool (NEP) exit polling operation has a multi-million dollar budget (Voter News Services, VNS, the forerunner to NEP, operated in 2000 on a budget of over $35 million; my understanding is that current costs are much lower but still in the millions). Note that in most states of interest, NEP will sample 20 to 50 precincts. As the scale of what the Current was attempting in a single county was in line with the exit poll that NEP conducts in each state.

I write this post not to beat up on the Current -- again, I give them credit for enterprise and transparency -- but to remind my media colleagues that all "exit polls" are not created equal. Not by a long shot.

Update: The cost statistic I cited for VNS from 2000 is accurate but potentially misleading. VNS was responsible for both exit polls and reporting final vote counts for every race (the latter function is now provided by the Associated Press). The costs also vary considerably between presidential and off-year elections.  Finally, the NEP exit operation still includes more than just exit polls, it also collects vote results at samples of key precincts and provides statistical modeling and analysis used to "call" races.