Polling Our Way To Confusion
[additional reporting by Sam Stein]
FiveThirtyEight's Nate Silver and Pollster's Mark Blumenthal are engaged in a fascinating debate about how pollsters can best frame questions about health care reform, and specifically, the public option.
Their back-and-forth reveals dramatic differences in polling philosophy, well worth pondering. It also lays bare a depressing reality: That constantly changing poll results are the desired product of a media that would vastly prefer serving as the narrator to a murky storyline -- replete with shifting winds, sturm und drang, and the endless saga of Who Is Up and Who Is Down --than incisively and aggressively penetrating an issue on behalf of the public interest.
Silver's position is that there is a "silver bullet" poll out there, and that by carefully considering the ingredients that go into the ask, a pollster can limit ambiguities and derive results that are precise. He lists five "essential ingredients" for a public option poll:
1. Make clear that the 'public option' refers unambiguously to a type of health insurance, and not the actual provision of health care services by the government.
2. Make clear that by "public", you mean "government".3. Avoid using the term 'Medicare' when referring to the public option.
4. Make clear that the public option is, in fact, an option, and that private insurance is also an option.
5. Ask in clear and unambiguous terms whether the respondent supports the public option -- not how important they think it is.
What a concept: don't conflate, don't confuse, level with people, and be complete. Silver is confident that pollsters can, with the right approach and the willingness to craft precise language, ask the right question.
Blumenthal (who, full disclosure, is a frequent contributor on these pages), by contrast, isn't much a fan of the notion that there's a perfect poll:
While I agree with some of Nate's observations, I have to disagree with his underlying premise. When it comes to testing reactions to complex policy proposals, I would rather have 10 pollsters asking slightly different questions and allowing us to compare and contrast their results than trying to settle on a single "perfect question" that somehow captures the "truth" of public opinion. On an issue as complicated and poorly understood as "public option," that sort of polling perfection is neither attainable nor desirable. In this case, public opinion does not boil down to a single number.
And yet neither Silver nor Blumenthal seem to desire the current state of coverage, where the media reports on a slaloming storyline, careering back and forth between daily winners and losers. Both assert that a "bottom line" exists. The debate here boils down between a statistical idealist (Silver) and a statistical skeptic (Blumenthal).
The Huffington Post recently got deep in the weeds on the issue of public option polling after an NBC/Wall Street Journal poll went out with an alteration in its own language:
Instead of asking whether people should be given a choice between a public and private plan -- as NBC/WSJ had done in its June 2009 survey -- the pollsters dropped the word "choice" in their July and August polls. In its place they asked whether people favored or opposed creating a public plan to compete with private insurers. Whereas two months ago, 76 percent of respondents said they felt it was either extremely or quite important to have a public option, in August that number was down to 43 percent.
"I think it's a very big deal to drop the word," said Wendell Potter, a former vice president at the insurance giant CIGNA. "This has been a strategy the industry has had for many years. They ask questions in many ways, knowing the way they are asking the questions will skew the result. Dropping the word choice is very important. It plays into some of the fears some of the people have been hearing lately, that the government would leave them without an option."
In subsequent email exchanges with the Huffington Post, Republican pollster Bill McInturff, who along with Democratic pollster Peter Hart coordinates the NBC polls, insisted that the use of the word "choice," prejudiced the survey and needed to be dropped. "There is a substantial difference between asking Americans whether they support having a choice of plans to asking what is a very different question, which is whether they support/oppose the concept," he wrote.
He also noted that pollsters for ABC and Washington Post had asked the same question about the public plan over a multiple-month period - which may be the most honest way to determine how opinion of the provision has evolved. Whereas in July 62 percent of respondents said they supported "having the government create a new health insurance plan to compete with private health insurance plans," by August that number had dipped to 52 percent.
"The NBC findings and the ABC/Post findings continue to suggest a closely divided electorate on this question which is of course much more in keeping with the significant emotions generated about this proposal on both sides of the issue," McInturff concluded.
But, for proponents of the public option, the sticking point is not whether respondents favor or oppose a government-run program, it is quite literally whether they want the "choice." The public option, as it's been drawn up, would only ever be an option.
Then again, "choice," as NBC's Chuck Todd noted to the Huffington Post, is a "trigger" word that few people oppose.
Moreover, given the mess of messaging coming from the White House, it's very hard to get clarity on the issue. President Barack Obama has always been more forthright and more consistent in his insistence that nobody would be forced out of a private health care option that they currently have and which they prefer. He's been far less clear on his support for the public option. This sows confusion: if the public option isn't an essential ingredient, where's the need to remind people that they won't be forced out of the private insurance that they may prefer?
More consistent messaging from the White House might have a beneficial effect on the way polls are conducted. To Silver, the idealist, the clarity would provide the means by which an exacting poll question could be crafted. To McInturff, it might allay the fear that central ideas were nothing more than semantic "triggers." Nevertheless, you will still have empiricists, like Blumenthal, who will shy away from anything that looks like they are taking a side in the debate, and who will seek out additional statistical yields to sift through, in search of a perfect objectivity to stand alongside subjective opinion.
The problem here is, policies have measurable merits and definable flaws, and while the Nate Silvers of the world will say that elucidating either will yield an exacting measure of public opinion, a Mark Blumenthal will tell you that this work is outside their jurisdiction. That would be for the best, if we had a press that was geared toward informing the public of policy merits, and that actually wanted to courageously render judgment on which side of a given issue enjoyed the greater share of truth. Unfortunately, the press, too, is geared toward keeping both sides in the game in order to craft a melodramatic political narrative. Pollsters who seek out a perfect objectivity over a precise truth feed that beast.
In short, this relationship between pollster and media yields an abundance of information about what people think, on any given day, at the expense of what can be known.
[Would you like to follow me on Twitter? Because why not? Also, please send tips to tv@huffingtonpost.com -- learn more about our media monitoring project here.]
Loading comments…
First Posted: 08-28-09 05:44 PM | Updated: 09-28-09 05:12 AM