The Harris "Wingnut" Poll: Could We Do Better?

My column
reviews the criticism of the Harris Interactive "wingnut" survey
released last week and suggests some ways to better measure some of the
beliefs it tested (taking up the dare issued via
Twitter by Markos Moulitsas). Please click through and read
it all.

One issue I did not mention in the column, and have not yet seen
raised elsewhere, is that the Harris survey asked the questions using a
"grid" or "matrix" format, something Harris Poll Chairman Humphrey
Taylor confirmed via email. That means that the questions and answers
were presented in a table (like the examples shown here).

While many online surveys rely on the grid format, it as an critical
shortcoming: It makes it easy for respondents who prefer not to think
hard about the questions to "straightline"" their answers. As such, this
aspect of the design may have increased the potential for the "aquiescence
bias" to have increased the percentage who said that some of the
statements were true.

For more on the shortcomings of grids, see this report
from Survey Sampling International and the ongoing criticism
from the anonymous market research blogger Research Rants.