This guest pollster contribution from Stan Greenberg is part of Pollster.com's week-long series on his new book, Dispatches from the War Room. Greenberg is chairman and CEO of Greenberg Quinlan Rosner.
I want to come back in my next post to the first half of Charles Franklin's piece where he raises legitimate issues about my characterization of media polls. But before that I want to develop a point he makes -- about order of questions -- because I actually have some new information on the subject and it is hard to find anyone interested in such issues.
When it comes to the vote, I have spent a lot of time assessing how to get people most comfortable with answering and to minimize the number of false undecided. In my experience, the closer the question to the start of the survey, the larger the undecided. There is a price in possible bias in introducing prior questions but if those questions reflect the broad political context in which the vote choice is being made, you can risk that. So, we will usually have a right direction/wrong track question, most important problem (either open-end or closed), a favorability or thermometer battery on political leaders, parties and organizations -- broadly distributed and balanced. Each election is a test of whether that structure produces unbiased estimates.
Our national presidential results for Democracy Corps or by Greenberg Quinlan Rosner closely resemble the national media polls, with a somewhat smaller undecided (though I haven't done the comparison on this issue).
Presidential approval is different. People are comfortable answering the question and at presidential level, not a lot of undecided once in office for a period. When I polled in the White House for President Clinton, we started by asking the job approval after the thermometer/favorability battery and right before the vote. With President Clinton, I discovered that asked at that point, his approval rating was higher than reported by other organizations. So, we conducted an experiment where half the respondents heard the job approval at the front of the survey and half heard it right after the favorability battery about Clinton and other political leaders. The implicit comparison led people to rate Clinton higher. Since job approval was the indicator most important to us, we moved up to the front in the survey.
We continued that practice during Bush's term and Democracy Corps' approval for Bush was higher than the norm -- and often cited by the White House. Bush's job approval, unlike Clinton's, likely fell if considered alongside other leaders or if the survey dealt with Iraq, the economy, health care or any other topic where is approval was probably even lower. I know this is obvious, but order really matters.
I agree that openness is the best route to sorting these issues -- including publishing the full surveys wherever possible.