Surveys got it wrong in key battleground states ― but how far were the numbers off? Theories about “shy Trump supporters” get a post-election reexamination. And one pollster offers a defense of non-election surveys. This is HuffPollster for Monday, November 14, 2016.
POLLS MISSED THE MARK IN HEAVILY WHITE, WORKING-CLASS STATES - Nate Cohn, Josh Katz and Kevin Quealy: “It was the biggest polling miss in a presidential election in decades…. [T]he state polls... systematically underestimated Donald J. Trump’s standing in the Upper Midwest and Northeast. His strength there was enough to make him the president. Few saw it coming. Mrs. Clinton led in nearly every high-quality survey of Minnesota, Maine, Pennsylvania, Wisconsin and Michigan…. All of these states have something in common: They have a large number of white voters without a college degree…. The states with a large number of white working-class voters tend to be somewhat less populous than the more diverse and well-educated states along the coasts…. Mrs. Clinton tended to outperform in big, liberal states with larger Hispanic populations. That was true in battlegrounds, like New Mexico and Nevada. It was also true outside the battlegrounds, in Illinois, California, New York and Washington. Over all, the two types of misses nearly canceled out in national polls. But Mr. Trump’s gains among white working-class voters were far more important, because those voters are overrepresented in the most important battleground states.” [NYT]
But some disagree that the polls were that far off - Sean Trende: “[A] close look shows that the 2016 [swing state] polls performed just as well as they did in 2012 – no better and no worse. You can see this by calculating the ‘mean absolute error,’ which measures how far away the polls were from the actual result, regardless of in which direction they were wrong. So if Hillary Clinton won a state by one percentage point, a poll average that showed the state tied would have the same error as a poll average that showed Clinton winning by two. To put it simply, it is the size of the miss that’s important. The mean absolute error is calculated by taking an average of the misses…. As you can see in the table below, the battleground states of 2016 had the same mean absolute error of 2.7 as the 2012 polls. (And if you look at only the states that were rated as tossups in 2012, the mean absolute error was 2.9 points, even worse than the tossups this year.)” [RealClearPolitics]
So which is it ― were polls worse than usual, or not? - The New York Times Upshot and RealClearPolitics differ in some critical ways that cause their assessments of error to be different. First, the states each is using to calculate error differ slightly. RCP is using all of its “battleground states,” whereas the NYT is using the “ten states closest to the national average with at least three polls.” That means the NYT includes Minnesota, which saw substantial polling error, in its 2016 calculation, while RCP does not. And RCP includes Arizona and Georgia, where there were lower errors than in Minnesota. Notably, the NYT’s analysis is not looking at the same states over time ― it is looking at those closest to the national average, which varies by year. And finally, poll selection matters. RCP doesn’t include all polls in its averages ― it excludes many internet polls ― and looked at its final averages for those polls. The NYT calculated misses based on an average of all polls in the last 3 weeks of the campaign. Both approaches are valid, but lead to different conclusions on how badly polls missed in 2016.
THERE MAY HAVE BEEN SHY TRUMP SUPPORTERS AFTER ALL - Elizabeth Connors, Samara Klar and Yanna Krupnikov: “To avoid ‘looking bad,’ some people avoid answering survey questions or, even worse, outright lie. Social desirability becomes even more powerful in a negative context: The more negative messages people receive about a certain group, the more likely they are to avoid publicly associating with that group. By the time the 2016 campaign was in its home stretch, the media culture surrounding the Trump candidacy had become significantly more negative….Could Trump’s social undesirability have mattered for pre-election polls? The challenge here is that people refuse to answer survey questions in a variety of ways. Hiding your views need not mean lying. It could mean saying that you’re undecided or simply not taking the poll at all. Right now, the data don’t tell us definitively what happened. That said, certainly there were more undecided voters in 2016 than 2012 — 15 percent vs. 5 percent as of late October. Meanwhile, exit poll data suggests an unexpected level of enthusiasm for Trump, and more subdued reactions for Clinton….Moreover, although some research suggested that ‘shy Trump voters’ were few and far between, other research did find that Republicans were more likely to deflect questions about voting for Trump. And we do not know yet whether some Trump supporters simply didn’t participate in polls at all.” [WashPost]
WEAK LOCAL ECONOMIES COULD HAVE HELPED TRUMP - Jed Kolko: “Donald Trump performed best on Tuesday in places where the economy is in worse shape, and especially in places where jobs are most at risk in the future. The relationship between economic issues and Trump’s support is not straightforward. Earlier analyses, based on pre-election polls, suggested that Trump’s backers were wealthier on average than Clinton’s. Actual election results show no correlation between counties’ September 2016 unemployment rates and their level of support for Trump — places with higher unemployment rates were no more likely to vote for Trump than those with lower rates. Unemployment, however, is a crude measure of a local economy.….Trump beat Clinton in counties where more jobs are at risk because of technology or globalization. Specifically, counties with the most ‘routine’ jobs — those in manufacturing, sales, clerical work and related occupations that are easier to automate or send offshore — were far more likely to vote for Trump.” [538]
WHY NON-ELECTION POLLING MATTERS - Nick Gourevitch (D), via Twitter: “I know everyone is frustrated w/ polling but I’d like to discuss importance of public opinion polls (not election polls) in a democracy. Public opinion polls are an important form of accountability on our government. They are a kind of check and balance. Public opinion polls help prevent our elected officials from pursuing policies completely at odds with the public’s desires….Election polls attempt to predict an outcome of an unknown population (who will vote in any given election). It’s a tricky endeavor. Public opinion polls of all adults, for example, are the implementation of science and statistics to measure the opinion of a population. They are based in theory and regularly stressed tested by organizations like Pew Research to ensure they match known benchmarks….What happens when President Trump pursues a policy that public opinion polls show is unpopular? Should the media dismiss polls showing this as unreliable because they missed the election result? Absolutely not. Take deportations. Many polls ranging from Pew Research to Fox News to NBC/WSJ show Americans oppose mass deportations. Let’s say Trump pursues this policy and says we should ignore what polls say because polls didn’t predict his presidency...The media would have a responsibility to push back on this narrative. Public opinion research would be a key proof point.” [Twitter]
HUFFPOLLSTER VIA EMAIL! - You can receive this daily update every weekday morning via email! Just click here, enter your email address, and click “sign up.” That’s all there is to it (and you can unsubscribe anytime).
MONDAY’S ‘OUTLIERS’ - Links to the best of news at the intersection of polling, politics and political data:
-Donald Trump spent 50 percent more time in the decisive battleground states than Hillary Clinton in the 100 days before the election. [NBC]
-Tim Meko, Denise Lu and Lazaro Gamio visualize the results of the election. [WashPost]
-Stan Greenberg (D) thinks Clinton should have focused more on the economy than on attacking Trump. [Democracy Corps (D)]
-Robert P. Jones explains how white Christian majorities in some states contributed to Trump’s win. [NYT]
-Doug Rivers analyzes YouGov’s performance in the 2016 election. [YouGov]
-Jay DeSart reviews how some political science forecasts got the election right. [Political Data Nerd]