Pollsters Raise Alarm: Inaccurate Polls May Be Impacting Campaigns
share this story
WASHINGTON - A remarkable bi-partisan group of campaign pollsters released an open letter this afternoon that assailed the "sometimes uncritical media coverage" of the "proliferation" of public pre-election polls that fail to disclose basic information about how they are conducted and that "have the capacity to shape media and donor reactions to election contests."
The authors of the letter -- 9 Democrats and 10 Republicans -- amount to a virtual "who's who" of campaign pollsters, the political consultants that conduct the opinion surveys sponsored by political campaigns for their internal use.
Their message is a bit unusual: At a time when political journalists and bloggers are busily scoring the accuracy of the final public election surveys, these pollsters called on the news media to judge the quality of polls based on "the professionalism with which they are conducted" rather than "their accuracy in the closing weeks of the election."
More specifically, the campaign pollsters urged journalists to hold public polls to disclosure standards of the American Association for Public Opinion Research (AAPOR) that call on pollsters to release details such as the exact wording of questions, the demographics of their samples, the methods used to draw their samples and interview voters and the response rates they obtain.
The letter explains the sort of "rigorous examination" that these pollsters have proposed:
Public polls differ on whether they release their likely voter screens, sample frames, demographics of the sample, on inclusion of cell phones, bilingual interviewing as appropriate, and even whether they asked other potentially biasing questions prior to the horse race. Some of these procedures are especially important in accurately polling hard to reach groups, particularly young voters, low income voters, African American voters, Latino voters, and others for whom English is a second language. The actual sample sizes in these groups, prior to any weighting of the data, are also elements in getting both the total numbers right, and in respecting the views of hard to reach populations.
The AAPOR standards address reporting of these elements because they are important in judging the quality and professionalism of polls, and in interpreting their results. We would urge extreme caution in coverage of polls that do not meet these standards and would urge judgment of polls by them rather than by apparent accuracy in the closing weeks of campaigns.
The letter does not specifically address the ongoing "Transparency Initiative" launched by AAPOR in 2009 to encourage routine disclosure of this sort of data by pollsters. So far, 70 news and survey organization have pledged their support, although that list includes none of the partisan pollsters that signed today's letter.
Should the campaign pollsters be held to the same standards they are proposing for media polls? Yes, says Republican Bill McInturff of the firm Public Opinion Strategies, one of the co-signers. "When campaigns release polls," McInturff said via email, "they tend to release only a handful of questions, but yes, if the pollster is asked by the press, then I believe the pollster should provide the entire question wording, demos, weights, and other basic information. And yes, I believe the press SHOULD exercise 'extreme caution' about a poll where these variables are not known."
AAPOR officials reacted positively. "AAPOR strongly supports adherence to its Code of Professional Ethics and Practice and AAPOR's Standards for Minimal Disclosure, and appreciates and endorses any efforts to further this goal," said current AAPOR President and Gallup Poll Editor-in-Chief Frank Newport via email.
"This looks like a salutary initiative that we should support," added Peter Miller, the outgoing AAPOR past president who launched their Transparency Initiative last year.
Longtime public pollster Andrew Kohut, president of the Pew Research Center, reacted cautiously. "I'm appalled by a lack of professionalism in a lot of political polling," Kohut said, but the statement "sounds as if accuracy is not a consideration" at all, something he would consider "unacceptable." Kohut agreed that pollsters should disclose more about their work, but disagrees with ignoring measurements of accuracy altogether.
Open Letter on the Public Polls - November 8
Loading comments… 
First Posted: 11/08/10 08:05 PM ET Updated: 05/25/11 07:10 PM ET