Journalists Should Stop Highlighting Individual Polls And Focus On Polling Averages

Readers are far less likely to dismiss these results on partisan grounds.
10/09/2017 01:24 pm ET Updated Oct 20, 2017
Marvin Gentry / Reuters

People need to pay more attention to polling averages and less attention to individual polls. There are two critical reasons for this. First, polling averages tend to be more accurate than individual polls in describing public opinion. Second, people are predisposed to dismiss individual polls altogether when the results suggest that they hold minority views or when their preferred candidate is losing. That is not the case with polling averages. Individuals are far less likely to dismiss these results on partisan grounds.

Although individual polls continue to dominate media coverage, the rise of polling blogs and data journalism have focused increasing attention on polling averages. HuffPost, the New York Times, and FiveThirtyEight provide polling averages for journalists to integrate into their stories. Methodological experts agree that polling aggregations produce a more reliable estimate of public opinion, at least when there is sufficient polling to generate an average (such as for presidential approval ratings or issues like health care).   

The tendency to believe polling averages over individual poll results follows a simple logic: an average of many measurements constitutes a richer and more reliable set of information than only one measure of the same thing. We engage in this sort of “aggregated evaluation” every day. Regardless of whether we do this intentionally or intuitively, we’re inclined to gather facts from multiple sources before making a decision. It is then reasonable to expect that the public would find poll averaging to be more credible (than individual poll results) and that this increased credibility could then be leveraged to lessen partisan biases in the processing of poll results.

To test this, we conducted a study of 1,263 individuals during the 2016 U.S. election. We presented these them with news stories reporting either that Hillary Clinton or Donald Trump was leading in the race for the presidency (for the methodological details and the news stories, see here). The stories were also either based on an individual poll result or presented an average of polls. It turns out that people’s partisan biases are so strong that they rejected the polling averages in the same way as singular poll results. When Clinton supporters saw information that Trump was ahead, they told us that neither the poll nor the polling average were particularly credible. Trump supporters did the same thing when they learned about a poll or an average where Clinton was leading. People were still determining whether the stories were true based on whether their candidate was ahead.

However, this finding does not fully capture what was happening at the individual level, as respondents’ education did have a different influence on their biased interpretation and responsiveness to the type of report.

When a respondent viewed an individual poll result, they were more likely to discredit the unfavorable result as inaccurate and this partisan bias was stronger if they were more educated. This is illustrated in Figure 1 where the higher values on the vertical axis (going from bottom to top) represent greater perceived accuracy and higher values on the horizontal axis (going from left to right) represent respondents with greater education. More educated individuals exhibited stronger bias, presumably because they can more easily find reasons not to believe the unfavorable messages they encounter. This replicates our previous findings. 

Kuru/Pasek/Traugott

On the other hand, although respondents presented with polling averages still reported that favorable results were more accurate than unfavorable ones, this bias shrunk among more educated respondents. When highly educated respondents viewed an unfavorable poll average, they reported that it was as just as accurate as a single favorable poll result. This shows that averaging served as a strong informational corrective for more educated respondents (Figure 2).

These findings show us the potential value of polling aggregations to counteract partisan biases. The results also suggest that in order to best communicate public opinion, journalists should refer to polling averages more frequently and include such information when presenting single poll results. It is also important to inform and engage news readers by highlighting the methodological strength of polling averages ― especially those with low education and less interest in the topic.

Ozan Kuru is a PhD Candidate (ABD) at Communication Studies and a Rackham Predoctoral Fellow at the University of Michigan.

Josh Pasek is an Assistant Professor of Communication Studies and Center for Political Studies Faculty Associate at the University of Michigan.

Michael Traugott is a Research Professor at the Center for Political Studies, the Institute for Social Research at the University of Michigan.

CONVERSATIONS