06/11/2013 06:12 pm ET

POLLSTER UPDATE: What The NSA Polling Doesn't Tell Us


Today we look at why two initial polls on the NSA surveillance controversy appear to tell different stories about where the public stands and ask, what if privacy 'hawks' don't trust pollsters to protect their privacy? And you want more Simon Jackman? You got him! This is the HuffPost Pollster update for Tuesday, June 11, 2013.

TWO NSA POLLS, TWO STORIES - Yesterday's update included results from a new Pew Research survey showing that a majority of Americans (56 percent) consider the NSA's tracking of phone records "acceptable." Late Monday afternoon, we published results from a new HuffPost/YouGov online poll showing a different result: "According to a new HuffPost/YouGov poll, 55 percent of Americans think collecting and analyzing Americans' phone records is 'an unnecessary intrusion into Americans' lives,' while 22 percent say that it is 'justified to combat terrorism.' By a 41 percent to 25 percent margin, a plurality of poll respondents said that collecting and analyzing Americans' phone records is an ineffective way of combating terrorism." But the two surveys were consistent in finding that most Americans are so far not closely following the story: Only 36 percent of respondents to the HuffPost/YouGov poll said that they had heard a lot about the NSA obtaining phone records of Verizon customers, while another 40 percent said they had heard a little and 24 percent said that they had heard nothing at all. Similarly, in the Post/Pew survey, just 27 percent of respondents said that they were following the story "very closely." [HuffPost]

Why the difference? - As the Pew Research Center's Michael Dimock recently argued, public inattention to a complex issue creates an environment in which relatively small differences in question wording can produce very different reactions. And in this case, the wording was different: "The Post/Pew question pointed specifically to a court order (albeit a secret one) and said that the program tracked 'millions of Americans.' The HuffPost/YouGov poll, on the other hand, noted in a previous question that the NSA was obtaining the records of all customers of Verizon, before asking if the program was 'justified to combat terrorism' or 'an unnecessary intrusion.'...The HuffPost/YouGov question asking whether the program was justified or an unnecessary intrusion also followed a question on whether tracking Americans' phone records was an effective or ineffective way of combatting terrorism, perhaps calling into question for respondents whether the program was was worthwhile, given its potential effectiveness. [Ibid]

Wording differences in Post/Pew poll - Columbia University Prof. Andrew Gelman: "At first glance, support for the surveillance seems slightly higher than before, with 51% supporting it in 2006, and 56% supporting it now. But look carefully at the questions: In 2006: 'secretly listening ... without court approval.' In 2013: 'getting secret court orders ...' So, more people support wiretapping now—-but the survey stipulates that the NSA got court orders. Sure, they're 'secret' court orders, but it means that a judge is somewhere in the loop. In contrast, the 2006 poll asked about extrajudicial wiretapping. On the other direction, the 2013 question refers to 'millions of Americans,' whereas the 2006 question asks about a more restricted class: 'people suspected of terrorist involvement.'" [The Monkey Cage]

PRISM polling may be different - Nate Cohn: "The Pew/Post poll did hint, though, that the public might not support PRISM—which includes email snooping—as much as the NSA's collection of phone records, which does not involve eavesdropping. Pew asked whether Americans thought the 'the U.S. government should be able to monitor everyone's email and other online activities if officials say this might prevent future terrorist attacks,' and found that 52 percent were opposed, compared to 45 percent in support. That's a net--22 points less support than the NSA's phone program, which was opposed by 41 percent of respondents." [TNR]

Twitter reactions to NSA polling:

DO PRIVACY 'HAWKS' PARTICIPATE IN SURVEYS? That last question by pollster Mike Mokrzycki is worth considering. All surveys require participation from respondents who share private attitudes and demographic details on the usually implicit promise that their confidentiality will be protected. Is it possible that the Americans most concerned about invasions of their privacy are less likely to participate in surveys? And if so, how would we know? This issue has been considered in depth by methodologists investigating whether such concerns might help explain a long term decline in survey response rates. Reviewing available experimental evidence for a chapter published in 2006, scholars Eleanor Singer and Stanley Presser found that concerns about privacy and confidentiality helped to reduce cooperation with the Census and other government surveys, but concluded that "the net effects of such attitudes, although statistically significant, are relatively small." [Advances in Telephone Survey Methodology]

Evidence of privacy 'bias'? - We put the question to Scott Keeter, director of survey research at the Pew Research Center, who has led a series of studies on the problem of declining response rates in surveys. "It's a reasonable theory," he replied via email, "but I am aware of only modest evidence in support of it." Keeter flagged the Singer/Presser review above, then summarized findings from Pew Research: "In our 2003 nonresponse study, we included a question that read, 'How much do you worry that computers and technology are being used to invade your privacy?' We didn't find any difference between the lower and higher response rate versions in answers to the question. (Question 13 here). Our 2012 nonresponse study found big differences between in estimates of sociability and volunteer activity from our survey and identical questions asked on the Current Population Survey (we found much higher levels on both variables), but it's not clear to me that those measures are capturing concerns about privacy, at least not directly. [Pew Non-Response studies from 2003 and 2012]

WHY DEMOGRAPHICS ALONE MIGHT NOT GIVE THE GOP A 2014 BUMP - Harry Enten: “Did Democrats do so poorly in 2010 because of poor turnout among their core constituencies: minorities and young voters? After all, the percentage of eligible voters that turns out drops by about 20pt between midterm and presidential years. Are most of those minority and young voters? If so, Republicans would have fared far worse in 2010 with presidential year turnout and will do well in 2014....Combining the impact of race and age, you'd be looking at a 3pt Republican gain between the 2008 and 2010 electorate. That still would have been enough for the Republicans to win more than 45 seats and win back the House. Keep in mind, though, the largest portion of the electorate that is 'minority' are youth. Putting together the age and race effects, therefore, may make the presidential and midterm electorates seem more different than they are...Pew data suggest that Republicans are actually not gaining an advantage; or if there is one, it's minimal. My own estimate is that it's probably about 2pt when looking at all the data. That's consistent with the idea that the current party affiliation differences between age groups has disadvantaged Democrats in midterm elections, while the differences in turnout by different racial groups likely don't have much of an effect." [Guardian]

GEORGE W. BUSH'S RATINGS IMPROVE - HuffPost: “For the first time since 2005, more Americans now view former President George W. Bush favorably than unfavorably, according to a Gallup poll released Tuesday. Forty-nine percent have a favorable view of Bush, while 46 percent view him unfavorably, the poll found. His ratings have risen by more than 10 points among both parties since he left office, with 84 percent of Republicans and 24 percent of Democrats now rating him favorably. At Bush's lowest point, in 2008, just 32 percent of Americans rated him positively, according to Gallup. Like most recent presidents, however, he saw his numbers rise after leaving the White House.“ [HuffPost]

Not a surprise:

NEWS ABOUT US - Some good news to report from the Pollster home office: Stanford political scientist Simon Jackman, who created the poll averaging model for HuffPost Pollster that correctly forecast the outcome of the presidential election in 51 out of 51 contests (all 50 states plus D.C.) is officially on-board to continue in that role through 2016. Look forward to seeing more of his work on the The Huffington Post! [51 out of 51]

TUESDAY'S 'OUTLIERS' - Links to more news at the intersection of polling, politics and political data:

-MassInc/WBUR poll finds Markey leading Gomez, 46 to 39 percent. [WBUR]

-Support for affirmative action hits a historic low. [NBC/WSJ]

-Nate Silver sees the domestic surveillance controversy dividing both parties in the 2016 primaries. [NYTimes]

-The American National Election Studies releases its time series data for 2012. [ANES]

-Seth Masket speculates about how the NSA uses call metadata. [Pacific Standard]

-A graphic designer redoes the PRISM slides. [SlideShare]

-Finding Paul Revere with metadata. [Kieran Heily via @AlexLundry]

Subscribe to the Politics email.
How will Trump’s administration impact you?