WASHINGTON -- Gallup, the polling firm that took a major hit last year for finding better results for GOP presidential nominee Mitt Romney than other pollsters found, offered a detailed mea culpa on Tuesday, citing a confluence of small issues that tipped its results rightward.
Over the final weeks of the 2012 campaign, Gallup's daily tracking poll showed President Barack Obama consistently trailing Romney, including a final survey that gave Romney a 1-point edge. Instead, Obama won by nearly 4 percentage points. While Gallup was far from the only polling firm to call the election wrong, its visibility and reputation, as well as the size of its error, made it one of the most notable.
"None of these factors are large, in and of themselves," said Gallup Editor-in-Chief Frank Newport, "in the sense that they are not moving the numbers 10 points, 5 points." However, each of the four nudged Gallup's numbers slightly in Romney's favor, making them collectively "significant enough," according to Newport, "that we think they made a significant difference in our overall estimate of who was going to win the presidential election last fall."
Here’s a brief summary of those issues, several of which The Huffington Post addressed earlier in more detail:
Misidentification of Likely Voters. Like most other media pollsters, Gallup attempts to determine which respondents are likely to actually vote in November, using a procedure developed in the 1950s that involves a battery of questions, such as how often they've voted in the past and how much attention they're paying to the election. Last year, this likely voter model moved Gallup's estimate of the margin separating Obama and Romney 4 points in Romney's direction.
"That, in and of itself," Newport said, "was at least 1 point more towards Romney than the average of other polls that were using some time of likely voter model." Thus, "on that sense alone," he concluded, "they moved things too far in moving the sample towards Mitt Romney."
While Gallup is continuing to investigate its likely voter procedures with a series of experiments in the fall 2013 state elections, it pointed out one big issue in 2012: that many Obama voters said they hadn't given much thought to the election, removing them from the likely voter pool even though they intended to cast a ballot. Gallup is researching whether it needs to majorly overhaul or even replace the way it identifies likely voters.
Under-Representation of Regions. Gallup also weights its data by a variety of factors, including broad geographical regions such as the Midwest and the South. But each region contains several time zones. Due to differing response rates, Gallup didn't interview enough people in certain time zones within some regions, effectively undersampling states that vote more Democratic.
Faulty Representation of Race and Ethnicity. As HuffPost first reported in June 2012, Gallup in recent years has used an unusual method to ask about race that distorted the racial composition of its samples when the data were weighted. Unlike most other pollsters, who ask respondents to select from a list of racial and ethnic categories, Gallup asked respondents whether they identified with each racial and ethnic group one by one. This led to a disproportionate number of people who said they were multiracial, and that in turn distorted the weighting procedure, effectively giving too much weight to some white voters.
Earlier this year, Gallup eliminated the yes/no racial questions and made some additional technical changes to its weighting procedures. As a result, Newport said, "we [now] do a better job moving our sample to the parameters we have as our gold standard ... for overall race representation."
Nonstandard Sampling Method. Before 2011, Gallup had selected phone numbers using random digit dialing, or RDD, which calls randomly generated numbers. This is the procedure that most national media polls have used for decades. As reported by HuffPost earlier this year, Gallup made a significant change in 2011, when it dropped the RDD methodology for its landline sample, using instead numbers randomly selected from those listed in residential telephone directories.
Gallup was alone among national pollsters in making this move, which promised to cut costs by eliminating unproductive calling to business lines and nonworking numbers. But the change came with a downside: Not everyone who has a landline has a listed number. Although Gallup's initial research indicated that cell phone calls would cover the difference, they didn't: The listed sample turned out to be older and more heavily Republican than the RDD sample.
Gallup is now in the process of switching back to RDD sampling for both landline and mobile phones, Newport said.
"Although in theory the use of the landline listed sample made sense ... it probably was a contributing factor to a skew of the sample that we felt was inappropriate, and therefore, we're making that change," he said.
In addition, Gallup also identified a number of factors it said didn't influence its results. Among them: the tracking poll format, the use of the name "Gallup," the race and gender of interviewers, the handling of third-party candidates, the order in which candidates' names were read, and the way undecided voters were nudged into picking a candidate.
But the firm said it's continuing to research these and other areas, using two 2013 elections -- the New Jersey and Virginia gubernatorial races -- as a sort of polling test lab. The results, which won’t be released to media, are intended to shed light on some of the questions plaguing Gallup individually and the polling industry as a whole.
"We think this is a perfect environment in the real world to see what's going on," Newport said. "It's very difficult to do retrospectively without an actual election to use."
Gallup's research later this year will include experimental surveys that attempt to validate the actual turnout of sampled voters in order to assess the accuracy of the likely voter selection process. These surveys will be designed to test a theory on which HuffPost previously reported, that many nonlikely voters were, in effect, screening themselves out -- by declining to participate in the survey.
"Does the fact that [respondents] have gone through the gantlet to the point where they are in our survey," Newport asked, "could that make a difference in terms of saying that they're already more likely to vote than the people who don't enter the sample?"
The ongoing research is being led by Michael Traugott, a renowned political scientist and survey methodologist at the University of Michigan, and several other academic experts in survey methods and statistics. Traugott will organize a fall seminar involving students in the Joint Program in Survey Methodology at Michigan and the University of Maryland, who will participate in the experiments' design. Newport confirmed that Gallup will make the "raw" respondent-level data available to scholars generally through the Roper Center Public Opinion Archives.
Also on HuffPost:
2012 -- Mitt Romney
Republican presidential candidate, Mitt Romney, speaks at the podium as he concedes the presidency on November 7, 2012 in Boston, Massachusetts. (Photo by Matthew Cavanaugh/Getty Images)
2008 -- John McCain
Sen. John McCain, R-Ariz., gestures to his supporters, while his wife, Cindy looks on during his concession speech at the Arizona Biltmore in Phoenix, Tuesday, Nov. 4, 2008. (AP Photo/Elise Amendola)
2004 -- John Kerry
Former Democratic presidential candidate U.S. Senator John Kerry (D-MA) stands on stage with his wife Teresa Heinz Kerry after delivering his concession speech at Faneuil Hall on November 3, 2004 in Boston, Massachusetts. (Photo by Chris Hondros/Getty Images)
2000 -- Al Gore
Democratic presidental candidate Al Gore leaves the voting booth after casting his vote at Forks River Elementry School in Elmwood, Tennessee on November 7, 2000. (TIMOTHY A. CLARY/AFP/Getty Images)
1996 -- Bob Dole
Republican presidential candidate Bob Dole lowers his head while making his concession speech to supporters at a Washington hotel, on Tuesday, Nov. 5, 1996. (AP Photo/J. Scott Applewhite)
1992 -- George H.W. Bush
U.S. President George Bush concedes the election on Nov. 3, 1992 after losing to President-elect Bill Clinton. (BOB DAEMMRICH/AFP/Getty Images)
1992 -- Ross Perot
U.S. independent presidential candidate Ross Perot delivers his concession speech on November 3, 1992 after Democrat Bill Clinton won the presidential election. (Photo credit should read PAUL RICHARDS/AFP/Getty Images)
1988 -- Michael Dukakis
Democratic presidential candidate Michael Dukakis wipes his upper lip during the first presidential debate with his opponent U.S. Vice President George Bush in Winston-Salem, N.C. on Sept. 25, 1988. (AP Photo/Bob Jordan)
1984 -- Walter Mondale
Defeated presidential hopeful Walter Mondale addresses supporters at night, Tuesday, Nov. 7, 1984 at the St. Paul Civic center, conceding to President Reagan. (AP Photo/Jim Mone)
1980 -- Jimmy Carter
U.S. President Jimmy Carter concedes defeat in the presidential election as he addresses a group of Carter-Mondale supporters in Washington, D.C., on Tuesday, Nov. 4, 1980. (AP Photo/Barry Thumma)
1976 -- Gerald Ford
President Gerald Ford speaks in the White House Press Room in Washington on November 3, 1976, conceding defeat to Jimmy Carter. (AP photo/ stf)
1972 -- George McGovern
Sen. George McGovern and his family in Sioux Falls, election night, Nov. 7, 1972 after he was defeated by Richard Nixon, and conceding the election. (AP Photo)
1968 -- Hubert H. Humphrey
Vice President Hubert H. Humphrey spaks at the Alfred E. Smith memorial dinner in Waldorf Astoria on Oct. 16, 1968 in New York. (AP Photo/John Lent)
1964 -- Barry Goldwater
A contact sheet of Republican senator Barry Morris Goldwater of Arizona concedes the 1964 presidential election to President Lyndon Johnson at a press conference held at his campaign headquarters at the Camelback Inn, Phoenix, Arizona, on November 4, 1964. (Photo by Washington Bureau/Getty Images)
1960 -- Richard Nixon
Vice President Nixon points to home-made sign at airport as he arrives in home state to cast his ballot on Nov. 8, 1960 in Ontario, California. (AP Photo)
1956 -- Adlai Stevenson
Sen. John F. Kennedy of Massachusetts talks with Democratic presidential candidate Adlai Stevenson on August 12, 1956 in Chicago. (AP Photo)
1952 -- Adlai Stevenson
Movie Actress Piper Laurie (left) is wearing a donkey head beauty spot on her cheek as she chats with Gov. Adlai Stevenson of Illinois, Democratic presidential nominee in Portland on Sept. 8, 1952. (AP Photo)
1948 -- Thomas Dewey
Dewey ran as the presidential candidate of the Republican Party in the elections of 1944 and 1948. (Photo by Fox Photos/Getty Images)
1944, 1948 -- Thomas Dewey
Thomas Dewey (1902 - 1971) Governor of the State of New York broadcasting over the 'Crusade of Freedom' radio. Dewey was the presidential candidate of the Republican Party in the elections of 1944 and 1948. (Photo by Keystone/Getty Images)
1940 -- Wendell Wilkie
Wendell Willkie, rehearses a report to the nation at a New York City radio station on Oct. 26, 1942. Willkie was President Roosevelt's personal representative, and his Republican opponent in the 1940 presidential elections. (AP Photo/Murray Becker)
1936 -- Alf Landon
Gov. Alf M. Landon, G.O.P. presidential nominee, voting in Independence, Kansas on Nov. 3, 1936. (AP Photo)
1932 -- Herbert Hoover
Herbert Hoover is shown leaving Madison Square Garden, Oct. 31, 1932 in New York City, after delivering his major campaign address before a crowd estimated at 22,000. (AP Photo)
1928 -- Alfred E. Smith
Governor Alfred E. Smith speaks in New York on Nov. 2, 1928. (AP Photo)
1924 -- John W. Davis
John W. Davis, Democratic nominee for President of the U.S., and his wife, are pictured on the estate of Charles Dana Gibson at Seven Hundred Acre Island in Dark Harbor, Maine on July 21, 1924. (AP Photo)
1920 -- James M. Cox
Democratic candidates for the presidency and vice-presidency of the United States, Governor James M Cox and Franklin Delano Roosevelt (1882 - 1945) are seen at the head of a nomination parade in Dayton, Ohio on Nov. 1, 1920. (Photo by Topical Press Agency/Getty Images)
1916 -- Charles Evans Hughes
1912 -- Theodore Roosevelt
Theodore Roosevelt during the progressive campaign of 1912. (AP Photo)