Just over half of those who support President Donald Trump said that the fictional “Bowling Green Massacre” shows why Trump’s executive order on immigration is necessary, according to a new poll out from Democratic pollster Public Policy Polling.
That makes a splashy headline, and it’s catnip for liberals who want to laugh at how stupid they think Trump supporters are. But it’s not good polling practice and should not be reported without substantial caveats about how the question was written and likely perceived by respondents.
First, most of us who read polls live in a world focused on media and politics, and we should get out of that mindset. The average American doesn’t spend hours of their day glued to Twitter and cable networks watching all the latest developments in Washington, D.C. Most are busy living their lives, caring for families, working non-political jobs or all of the above.
In that frame of mind, how many Americans have probably heard of the Bowling Green Massacre, or know it’s a fake thing Kellyanne Conway made up last week? We don’t know. The poll didn’t ask that question. The question wasn’t even about whether people believed the fictional massacre happened ― that was simply asserted as a fact.
What the question did ask about was whether respondents agreed that a fake event ― presented as a factual event ― justifies a policy that many Trump supporters already support. Of course many supporters were going to agree with that statement, even if they weren’t aware that the Bowling Green massacre was fiction.
Not knowing about the issue doesn’t make people stupid, either. The pace of news in the last few weeks has been extremely fast. People with nonpolitical lives can’t be expected to keep up.
There’s considerable research on how average people answer poll questions when they might not really know what the question refers to. Some will admit that they don’t know the answer ― as 20 percent of the whole sample and 23 percent of Trump supporters did in this case. But many will think they should have an answer, and say the first thing that comes to mind. This is part of why polling on specific policies is difficult ― people often haven’t given issues a lot of thought, but when prompted, they will make up an opinion.
Research also shows that when you ask people to agree or disagree with something, they are more likely to agree if they don’t have a solid opinion. This is called “acquiescence bias,” and it’s why many pollsters shy away from yes/no or agree/disagree types of questions.
So PPP’s results likely don’t tell us anything about whether people think the Bowling Green massacre was a real event, as some are interpreting the poll. The question directly tied the Bowling Green Massacre to Trump’s executive order and asked it in an agree/disagree format that would lean Trump supporters toward agreeing.
The 10 or so media outlets covering this question should know better. PPP has a history of asking questions designed to make certain segments of the population look like they believe in fictional things. In late 2015, they asked about bombing the fictional Disney kingdom of Agrabah as if it were a real place. But many media outlets picked that story up too.
It’s even worse now. Many of Trump’s supporters don’t believe mainstream media, so to report on a poll question and basically say, “Haha, look how dumb these people are!” when the question was a total setup only exacerbates the problem that media has in the Trump era.
To assert a fake event as fact in a poll question only perpetuates falsehoods, and the Trump team doesn’t need any help in that area.