WASHINGTON -- If you care about politics and spend a fair amount of time online, odds are good that sometime between 5 p.m. and 7:30 p.m. Eastern time on Tuesday, you will encounter someone sharing leaked "exit poll" numbers that purport to tell you who will win various battleground states.
It may be tempting to assume the numbers will tell you who is going to win the presidential election.
They probably won't.
Hard as it may be, you should try to ignore them, at least until the polls close. And even then, take the underlying vote estimates with big grains of salt.
But let's first stipulate that once weighted to match the actual outcome, exit polls are an incredibly valuable resource. Conducted every two years by Edison Research for the National Election Pool partnership of ABC, CBS, CNN, Fox, NBC and the Associated Press, the exit polls are easily the best measure available of who voted for each candidate and why.
So it was for good reason that many political junkies expressed disappointment when news spread that the National Election Pool will only conduct complete exit polls in 31 states this year, down from the usual 50. In recent years, these surveys have become far more expensive, as the rise in early voting created the need for parallel telephone surveys to measure the preferences of early voters.
But as tools to predict the outcome of close races just before the polls close, they are blunt instruments at best. Here's why:
First, an exit poll is just a survey. Like other polls, it is subject to random sampling error, so differences of a few percentage points between the candidates in any given state sample are not terribly meaningful.
Second, the networks almost never "call" truly competitive races on exit poll results alone. The decision desk analysts require very high statistical confidence (at least 99.5 percent) before they will consider calling a winner (the ordinary "margin of error" on pre-election polls typically uses a 95 percent confidence level). They usually only achieve that confidence for relatively close races after the exit pollsters obtain the actual vote results from the randomly selected precincts at which interviews were completed (and from other larger random samples of precincts) and combine all of the data into some very sophisticated statistical models.
Even then, if the models project that the leading candidates are separated by just a few percentage points, as pre-election polls suggest they will be in all of the key battleground states, the networks will usually wait until nearly all votes are counted to project a winner.
Third, the initial results of the exit poll interviews have had frequent problems with non-response bias, a consistent discrepancy favoring the Democrats that has appeared to some degree in every presidential election since 1988. Usually the bias is small, but in 2004 it was just big enough to convince millions of Americans who saw the leaked results on the Internet that John Kerry would defeat George W. Bush. It didn't work out that way.
The resulting uproar led the networks, beginning in 2006, to hold back the data from their news media clients in a sealed quarantine room on Election Day until 5 p.m. Eastern time. The quarantine means that any numbers purporting to be "exit polls" before 5 p.m. are almost certainly bogus.
To try to minimize the potential errors, the networks often weight the first official tabulations they post on their websites to an estimate of the outcome (called the "composite") that combines responses to the exit poll interviews with the averages of pre-election polls (like those reported by HuffPost Pollster).
But that process is imperfect and does not remove either the random error or the initial statistical bias that often favors Democrats. Four years ago, on Pollster.com, we gathered all of the official tabulations posted as polls closed and extrapolated the underlying estimates of the outcome for each state. When later compared against the final vote counts in each state, we found that the initial estimates had overstated Barack Obama's margins by an average of 4.7 percentage points.
Does that mean that we can just subtract four or five points from Obama-minus-Romney margin and get a more precise estimate of the outcome? Nope. First, 4.7 was an average. The errors in individual states varied widely from 16-point overstatement on the margin favoring Obama to a 5.5-point error favoring John McCain, with misses spread across the spectrum in between.
Also, there is no guarantee that the 2008 errors will repeat at the same magnitude or that the exit pollsters have not made some adjustment this year to correct for past problem. Keep in mind that these issues do not lead to missed calls, both because the decision desk analysts are aware of them and because they have the ability to estimate and correct the errors in near-real time, as they systematically compare the incoming vote returns from the sampled precincts to the exit poll responses gathered from them.
Those of us seeing leaked data, however, see neither the running calculations of the precinct errors nor the levels of statistical confidence associated with the vote numbers. We see only precise-looking percentages and are oblivious to the potential for error.
As one pundit put it four years ago, the exit polls "have become crack cocaine for political junkies looking to score on Election Day." We would be better off, he said, if we relied on the exit polls for "their original purpose, explaining who did what and why, rather than trying to forecast what will be widely known anyway in just a few hours."
2012 -- Barack Obama
U.S. President Barack Obama waves to supporters following his victory speech on election night in Chicago, Illinois on November 6, 2012. (JEWEL SAMAD/AFP/Getty Images)
2008 -- Barack Obama
Nov. 4, 2008: U.S. president-elect Barack Obama waves at his supporters during his election night victory rally at Grant Park in Chicago. (JEWEL SAMAD/AFP/Getty Images)
2004 -- George W. Bush
In this Nov. 3, 2004 file photo, President George W. Bush and first lady Laura Bush salute and wave during an election victory rally at the Ronald Reagan Building and International Trade Center in Washington. (AP Photo/Charles Dharapak, File)
2000 -- George W. Bush
U.S. Republican presidential candidate and Texas Governor George W. Bush casts his vote in Austin, Texas on November 7, 2000. (PAUL RICHARDS/AFP/Getty Images)
1996 -- Bill Clinton
President Bill Clinton, wife Hillary and daughter Chelsea wave to supporters in front of the Old State House during an election night celebration in Little Rock, Ark. on Tuesday, Nov. 5, 1996. (AP Photo/David Longstreath)
1992 -- Bill Clinton
Bill Clinton and Al Gore celebrate in Little Rock, Arkansas after winning in a landslide election on November 3, 1992. (AP Photo)
1988 -- George H. W. Bush
President-elect George Bush and his family celebrate his victory on November 8,1988 at the Brown Convention Center in Houston. (WALT FRERCK/AFP/Getty Images) <em><strong>CORRECTION:</strong> An earlier version of this slide was titled "George W. Bush." It has been fixed.</em>
1984 -- Ronald Reagan
President Ronald Reagan gives a thumbs-up to supporters at the Century Plaza Hotel in Los Angeles as he celebrates his re-election, Nov. 6, 1984, with first lady Nancy Reagan at his side. (AP Photo/File)
1980 -- Ronald Reagan
President-elect Ronald Reagan and wife Nancy wave to well-wishers on Tuesday, Nov. 4, 1980 at Century Plaza Hotel in Los Angeles after his election victory. (AP Photo)
1976 -- Jimmy Carter
Democratic presidential candidate Jimmy Carter embraces his wife Rosalynn after receiving the final news of his victory in the national general election on November 2, 1976. (Photo by Hulton Archive/Getty Images)
1972 -- Richard Nixon
U.S. President Richard M. Nixon meets at Camp David, Maryland, on November 13, 1972 to discuss the Vietnam situation with Secretary of State Henry A. Kissinger (L) and Maj. Gen. Alexander M. Haig Jr.(R), Deputy Assistant to the President for National Security Affairs. (Photo by AFP PHOTO/NATIONAL ARCHIVE/Getty Images)
1968 -- Richard Nixon
President-elect Richard M. Nixon and his wife, Pat, were a picture of joy at the Waldorf-Astoria Hotel in New York, Nov. 6, 1968, as he thanked campaign workers. At left are David Eisenhower, Julie Nixon's fiance, Julie and her sister Tricia at center. (AP Photo)
1964 -- Lyndon Johnson
President Lyndon Johnson proves he's a pretty good cowhand as he puts his horse, Lady B, through the paces of rounding up a Hereford yearling on his LBJ Ranch near Stonewall, Texas, on November 4, 1964. (AP Photo/Bill Hudson)
1960 -- John F. Kennedy
Caroline Kennedy peeps over the shoulder of her father, Senator John F. Kennedy, as he gave her a piggy-back ride November 9, 1960 at the Kennedy residence in Hyannis Port, Mass. It was the first chance president-elect Kennedy had to relax with his daughter in weeks. (AP Photo)
1956 -- Dwight D. Eisenhower
President Dwight D. Eisenhower and Vice President Richard Nixon salute cheering workers and Republicans at GOP election headquarters in Washington, November 7, 1956, after Adlai Stevenson conceded. (AP Photo)
1952 -- Dwight D. Eisenhower
President-elect Dwight Eisenhower and first lady-elect Mamie Eisenhower wave to the cheering, singing crowd in the Grand Ballroom of the Hotel Commodore in New York City on Nov. 5, 1952 after Gov. Adlai Stevenson conceded defeat. (AP Photo/Matty Zimmerman)
1948 -- Harry S. Truman
U.S. President Harry S. Truman holds up an Election Day edition of the Chicago Daily Tribune, which, based on early results, mistakenly announced "Dewey Defeats Truman" on November 4, 1948. The president told well-wishers at St. Louis' Union Station, "That is one for the books!" (AP Photo/Byron Rollins)
1944 -- Franklin D. Roosevelt
President Franklin Roosevelt greets a young admirer as he sits outside his home in Hyde Park, N.Y., on election night, November 7, 1944. Behind him stands his daughter, Mrs. Anna Roosevelt Boettinger and the first lady, Eleanor Roosevelt. (AP Photo)
1940 -- Franklin D. Roosevelt
American President Franklin Delano Roosevelt (1882 - 1945) speaking to a crowd of 25,000 at Madison Square Garden in New York on Nov. 8, 1940, before his sweeping re-election for a third term. (Photo by Fox Photos/Getty Images)
1936 -- Franklin D. Roosevelt
The Republican Governor of Kansas and presidential candidate, Alfred Landon (1887 - 1987) greeting the American President Franklin Delano Roosevelt (1882 - 1945) (seated) prior to the presidential elections. Future United States President Harry S. Truman can been seen in the background. (Photo by Keystone/Getty Images)
1932 -- Franklin D. Roosevelt
Governor Franklin D. Roosevelt of New York at his Hyde Park, N.Y. home November 6, 1932, seen at the conclusion of the arduous months of campaigning following his presidential nomination in Chicago. (AP Photo)
1928 -- Herbert Hoover
President-elect Herbert Hoover is seated at a table with wife, Lou, and joined by other family members on Nov. 9, 1928. Standing from left: Allan Hoover; son; Margaret Hoover, with husband, Herbert Hoover, Jr.,at right. Peggy Ann Hoover, daughter of Herbert Hoover Jr., sits with her grandmother. (AP Photo)
1924 -- Calvin Coolidge
U.S. President Calvin Coolidge and first lady Grace Coolidge are shown with their dog at the White House portico in Washington, D.C., on Nov. 5, 1924. (AP Photo)
1920 -- Warren Harding
Senator Warren Harding, with wife Florence and his father George, shown on Aug. 27, 1920. (AP Photo)
1916 -- Woodrow Wilson
Surrounded by crowds, President Woodrow Wilson throws out the first ball at a baseball game in Washington in this 1916 photo. (AP Photo)
1912 -- Woodrow Wilson
Woodrow Wilson (1856 - 1924), the future American president, casts his vote while Governor of New Jersey, on Nov. 14, 1912. (Photo by Topical Press Agency/Getty Images)