Donald Trump’s relationship with the polls has always been a fraught affair, largely dependent on how he happens to be doing in them.
During the primaries, when surveys showed Trump ahead, he couldn’t stop talking about his polling numbers. Earlier in the summer, when nearly every survey of the race showed him behind, his supporters instead embraced poll denialism.
And this week, in the aftermath of a presidential debate performance widely seen as moderately disastrous, Trump tweeted this:
First of all, this is not true. Online polls from Morning Consult and YouGov both showed Democratic nominee Hillary Clinton “winning” the debate, as did snap polling from CNN and the Democratic firm PPP.
Furthermore, what sets those polls apart from the ones Trump is citing is that they’re actually scientific.
Scientific polls can be conducted by phone, via online panels or some other way, and use a mix of sampling and weighting to make their numbers representative of the larger population whose opinions they’re measuring ― whether that’s all adult Americans, or just likely voters. Recent changes in technology have complicated that process, but the underlying principle remains basically the same.
That’s why, even if a scientific poll reaches relatively few people, it can accurately depict the opinions of a much larger group.
In contrast, reader polls, like those Trump cites in the tweet below, make no such attempt to weight their responses or to represent anything beyond the number of people who happen to have clicked on them.
As tools to accurately gauge public sentiment, such unscientific polls fail on a number of levels. The people reading any particular website aren’t representative of the public at large. Those who take the time to read a particular story ― and to weigh in on it ― are even less so.
For a quick visual example of why that’s a problem, let’s look at some of The Huffington Post’s own surveys. HuffPost partners with the polling firm YouGov to conduct scientific polls using an online panel. When we report the results of those surveys, we’ll also often include a reader poll in the story as well.
Those reader polls are just for fun ― the results aren’t actually used for anything. But it does give us the opportunity to compare the actual poll’s results, which are representative of the American public, to the reader poll’s results, which are representative of ... well, the people who took the poll.
HuffPost readers who took a reader poll in one election story, for instance, were twice as likely as the average American to be “extremely enthusiastic” about the election:
Respondents to another reader poll were far more likely than the public as a whole to say they’d fought with their friends over the presidential race.
And, for whatever reason, readers who answered this Thanksgiving poll were rather notably biased toward stuffing.
And that’s without getting into what’s possibly the worst problem with reader polls ― there’s often nothing to prevent people from voting multiple times.
That leaves them vulnerable to intentional manipulation by people with a vested interest in the outcome ― a fact that some Trump fans took full advantage of. (For a similar example, see this tale of how the internet nearly named a research ship “Boaty McBoatface”).
Trump isn’t the only one in his circle who seems to be confused ― or eager to blur the lines ― over what constitutes an actual poll. His campaign manager, Kellyanne Conway, tweeted a link to a Daily Mail story referencing reader polls. As a pollster herself, Conway almost certainly knows better. She deleted the tweet a few minutes later.
Fox News was also forced to circulate a memo after several of the network’s hosts, including Sean Hannity, cited unscientific polls about the debate.
“News networks and other organizations go to great effort and rigor to conduct scientific polls — for good reason,” wrote Dana Blanton, Fox’s vice president of public opinion research, according to Business Insider. “They know quick vote items posted on the web are nonsense, not true measures of public opinion.”
Handy rule of thumb:
Editor’s note: Donald Trump regularlyincitespolitical violence and is a