01/23/2009 12:10 pm ET Updated May 25, 2011

Why Pollsters Manipulate Public Opinion

Two recent polls, one by Gallup and the other by CNN, illustrate how easy it is for pollsters to manipulate public opinion into something different from what it really is.


The Gallup poll, Jan. 6-7, 2009, attempted to measure the public's reaction to a federal government stimulus package, with the question phrased as follows:


Do you favor or oppose Congress passing a new 775 billion dollar economic stimulus program as soon as possible after Barack Obama takes office?


In almost the same time period, the NBC News/Wall Street Journal poll also attempted to measure public opinion about the stimulus package, with a question that provided for a "don't know" option:


Do you think that the recently proposed economic stimulus legislation is a good idea or a bad idea? If you do not have an opinion either way, please just say so.


The results are shown below:


0901_16 CNN Manipulating public opinion (Econ stim pack).pngThe major difference, of course, is in the percentage of people who don't have an opinion - Gallup says just 11 percent, while NBC/WSJ says almost three times that number.


The margin in favor of the stimulus package is virtually identical in the two polls, 16 and 17 percentage points, but instead of being able to report a majority of Americans in favor, NBC and the Journal had to report that a "plurality" of Americans were in favor, with a substantial portion of the public ambivalent or unengaged. Gallup, by contrast, could report (although erroneously) that a majority was in favor.


It's not always the case that the margin in favor of a proposition is always the same in both ways of measuring public opinion, as is illustrated in the following case. When CNN wanted to discover whether the public was copasetic with Democratic control of all three branches of government, it asked a forced choice question (Nov. 6-9, 2008):


As you may know, the Democrats will control both the Senate and the House of Representatives, as well as the presidency. Do you think this will be good for the country or bad for the country?


Again, coincidentally, another polling organization, Associated Press/GfK Roper Public Affairs and Media asked a similar question at virtually the same time (Nov. 6-10, 2008), though this question allowed for a middle position:


As you may know, the Democrats will now control the House of Representatives, the Senate and the presidency. Do you think it good for the country, bad for the country, or does it not really make a difference that the Democrats now control the House, the Senate and the presidency?


The results of the two polls show two very different publics:


0901_16 CNN Manipulating public opinion (Dems control branches).pngWhile CNN reports a large majority of Americans in favor of Democratic control, by a 21-point margin, the Associated Press finds a small plurality in favor (just an 8-point margin) and about a quarter of the public either saying the situation doesn't matter or not expressing an opinion.


In this case, both polling organizations deliberately manipulated their respondents to come up with an opinion (even if, in the case of the AP/GfK poll, to say the issue didn't matter) by giving them information up front. Why did they need to tell respondents that the Democrats controlled all three branches? Why not find out how many people knew that, and then - among those who knew it - ask whether it was good or bad, didn't it make a difference, or didn't they have an opinion?


But the major media pollsters are generally not interested in realistic measures of public opinion. On the matters discussed here, Gallup and CNN clearly do not want to report how many people don't have an opinion or might want to take a valid middle position on the issue. Instead, these pollsters believe it's more interesting to create a "public opinion" that reflects a highly engaged and decisive public.


For CNN to say that 97 percent of Americans believe Democratic control of the government is either "good" or "bad," and for Gallup to claim that nine out of ten Americans have an opinion about the stimulus package, may fit their journalistic needs - but they know, and we know, it's simply not true.