Exit Poll Mania -- Beware

Read more Super Tuesday coverage on HuffPost

These early exit polls, courtesy of Open Left, could lead an Obama fan to faint:

Here you go. I have no idea how reliable these are. ...If these are real, Obama has thrashed Clinton today.
Georgia: Obama 75, Clinton 26
Connecticut: Obama 52, Clinton 45
Illinois: Obama 70, Clinton 29
Alabama: Obama 60, Clinton - 37
Delaware Obama 56, Clinton 42
Massachusetts: Obama 50, Clinton 47
Missouri: Obama 50, Clinton 45
Tennessee: Clinton 52, Obama 41
New York: Clinton 56, Obama 42
New Jersey: Obama 52, Clinton 47
Arkansas: Clinton 71, Obama 26
Oklahoma: Clinton 61, Obama 30
Arizona: Obama 51, Clinton 45

But, as a former John Kerry staffer from 2004, exit polls make me beyond anxious. On Election Day 2004, early exit polls had Kerry beating Bush, 50% to 48%. And we all know what happened. So as we head into the race-calling hours, with polls that will no doubt be tighter than the ones above, I'm dusting off my statistics, courtesy of my fantastic statistics teacher Deb Hughes Hallett. The horse race coverage is a dangerous thing. Take my home state in early exit polls: "Massachusetts: Obama 50, Clinton 47."

A single digit lead like this means it's too close to call, because of the margin of error. The margin of error goes like this (I found a great example online, at Robert Niles' blog, which I have adapted for this race). Say the poll has a 4% margin of error, which would be reasonable for a small sample. "That means that if you asked a question from this poll 100 times, 95 of those times the percentage of people giving a particular answer would be within 4 points of the percentage who gave that same answer in this poll.

With a 3 point lead and a margin of error of 4%, there is no reasonable statistical basis for claiming that Obama will win over Hillary in Massachusetts.

More on the MoE:

A poll of 250 votes gives the largest margin of error, about 6.2%. For a poll of 450 voters, the margin of error is about 4.6%.

What to do with the margin of error: If the difference between the two candidate's polls is less than the margin of error, you should say the race is too close to call. (Reason: The difference you are seeing could easily be the result of sampling variation in the poll.)

If the difference is more than twice the margin of error, you can say that the person who is ahead is likely to win. (In this case the difference is large enough that it is unlikely to be the result of sampling variation in the poll.)

What if the poll results are between one and two margins of error apart? This is the grey area, where it is possible that the candidate with lower poll numbers will actually win. The chance of this goes down as the difference between them gets closer to two margins of error. Let's assume that 450 people were polled: The margin of error is 4.6%. If there is a 9% difference between candidates, the top candidate probably will win. If the difference between candidates is less than 4.5%, the race is too close to call. Between 4.5% and 9%, you could to say something like "X appears to be ahead at the moment, but keep watching the polls."

Any statisticians in the audience? Am I missing anything important?

I'll be heading to Romney HQ in Boston soon -- I don't think it will be quite a party....

Read more Super Tuesday coverage on HuffPost

Comments

[stellanoir]

Riddle me this. . .?

How come exit polls were instituted in order to safe guard the integrity of voting processes and have been successfully used around the world with nominal margins of error for a very long time ?

Until we idiotically started using these nefarious machines in this country, they worked quite well here as well.

Since they've failed in recent years, all the punditry has come up with these lame theories about disingenuous voters, and the inaccuracies of pollsters, and scarcely few scrutinize the multiple insecurities of the vote tallying software (even with the optical scanners which generate paper which is never subject to review) or their manufacturers and technicians.

It's the machines. Please refrain from blaming the people and the pollsters.

Thanks in advance.

Have a nice day.

[nomobull]

one word swiftboating

[kylie]

It isn't exit polling that concerns me-and Kerry should and did win Ohio but we know what happened with Diebolds, voter suppression, fliers giving misleading address, date of election, missing ballots. Sound all too familiar?

What the voting public needs to be concerned about is voter "discrepancies" that happened in Florida, Ohio, New Mexico, etc in elections past and, we need to find out how today's primaries fared, too.

New Mexico has not been called-did not like the fact in '04 Richardson would not allow recounts, paper trails, and it was a murky election.

Now I'm concerned about the New Mexico numbers-Richardson helped Bush get in and he is friendly with the Clintons.

Diebolds need to go, people need to stop listening to the pollsters, we need to hear the political hosts/pundants with a grain of salt, and we all need to read, learn/educate ourselves about the candidates and Vote.

I'm a Californian who is definitely for Obama! One of millions!

[mwpeters]

After poking through some of the results, it looks to me like exit polls have another great weakness.

It appears very large percentages of vote in critical contests were pre-cast by absentee ballot. Look at the poll numbers for marginal candidates who dropped out a week ago or more. In many, many, cases I've spotted votes for Edwards (and occasionally also for Giuliani) that were near or over 10%. I think to get to that kind of result you'd need something like 20-30% of ballots being cast more than a week ago.....

If exit polls miss half the voters and are polling voters after voter sentiment may have shifted.

[Listen2me]

"We all know what happened" you say.

Well, WE know, as the comments from other posters have made clear. The 2004 election was STOLEN.

The people who stole it know they stole it. The people whose votes didn't count know it was stolen.

The only people who DON'T KNOW WHAT HAPPENED are the fools in the Democratic party leadership. Even with their supposed control of Congress now, they have done worse than nothing to restore fair and honest elections in this country.

Which leaves the door wide, wide open for the SAME scenario committed by Rove's Recidivists once again. And Rove, who the Democrats OUGHT to have put in jail for contempt of Congress, gets to be a television pundit, making huge piles of money and peddling a bunch of lies, while laughing his ass off at the stupidity, timidity, and gullibility of the Democratic party leadership and candidates.

[VoteforOCD]

Following exit polls closer this year than I have in the past, my memory may be failing me. Exit polls are worth shit. Exit polls give you the option to vote for Hillary and tell someone "Oh yes, I voted Obama. I'm color blind." Likewise, it seems that Obama does well in caucuses.

Obama supporters need to get off their asses and get to the polls if he's to have any chance. I've seen enough elections to know that those who have something to lose turn out, those trying to build something do not. Elections mean more to the Hillary supporters.

They see kids and blacks and some white men turning out for Obama. They know where the polls are and they have the discipline to show up. It's been the Clinton / Rove strategy for the last 4 elections. Scare the base, then tear down the opponent. Sadly, it works. To them the end justifies the means. Divide and conquer.

[humblediplomat]

According to author Marc Crispin Miller, in his book Fooled Again or something like that, he says the odds of the exit polls being so wrong in so many places in 2004 were more than 1 in a million.

In a normal statistical sample, they could be wrong by the margin of error, but when in every case they are wrong in the same direction (towards Bush), there is something else going on.

[tinkaboutit]

Yes Morra Aarons-Mele, you ARE missing something. Privately owned digital voting companies count our votes in secret. Exit polls are extremely accurate, and the declared results are not. Now all we need is the media to do its job and point the finger at the obvious vote fraud.

[tinkaboutit]

Exit polls are EXTREMELY accurate. FAR more accurate than polls before the election. They are a great way to verify an accurate vote count. In America the privately owned voting companies like Diebold counts our votes in secret. The media plays along with the digital vote fraud by covering up discrepancies between exit polls and announced results. In the Ukraine there was less of a discrepancy than in Ohio in 2004, and 200,000 people hit the streets and turned around the crooked election: http://www.cnn.com/2004/WORLD/europe/11/23/ukraine.1110/

Of course in America we are too lazy, gullible and cowardly to deserve a real democracy.

[NetWeasel]

I seem to remember that in 2004, when everyone said that the "exit polls MUST have been wrong," they STILL held every bit of it as gospel EXCEPT the Presidential voting numbers.
Percentage of blacks for Kerry, percentage of women for Bush -- all of those (and other demographic polling numbers) were apparently correct, the way they were being talked about.
Strange, huh?

[research]

Funny how we use any differences between exit polls and recorded votes to indicate election fraud, except in our own country.

[Jpol]

Sorry, but the exit polls were correct. It was the results (stolen votes) that were incorrect.

[research]

2004 the exit polls showed kerry won. So the MSM retroactively changed the exit poll numbers to match the recorded vote.

[donjoe4]

This is all a waste of time...Bush and Chaney arn't leaving..Martial law and takeover by November...you don't think a little thing like Democracy is going to stop them do you???

[MelindaPost]

Yes. What happened is that Kerry folded, chose not to make a fight of the stolen election.