First, let me say a quick but huge thank you to Charles
Franklin for his frequent posts over the last two weeks as I took much needed
holiday break and attended to two days of AAPOR
meetings last week. I'm glad he kept things
busy here while I lived the slacker life.
I should be back to a more regular schedule this week, with much to
catch up on.
While we are on the subject of the American Association for
Public Opinion Research, I want to say a quick word about the organization's
academic journal, Public Opinion
Quarterly, which last week released a special edition on "Non-Response Bias
in Household Surveys." For the
non-pollsters among you, "non-response bias" is the technical term for the
error that can result [especially] when response rates are low and those that respond to a
survey differ from those who do not [but it can result even when response rates are high; see the comment from Joel Bloom below].
The POQ special edition includes articles and research from the most
respected authorities on this subject, and best of all, the editors have made electronic access to this
edition completely free.
One of the ideas that we try to stress here on Pollster is
that polls are subject to all sorts of potential error not captured by the
so-called "margin of error." The study
of non-response may be a bit arcane to ordinary political junkies, but if the
POQ Special Edition proves anything, it is that academic survey researchers
have been studying it for quite a long time.
Consider this summary from the introduction by Eleanor Singer, the
editor of the special edition:
Concern about survey nonresponse is of course not new. Smith (2002, pp. 27-28) notes that "early research extends back to the emergence of polling in the 1930s and has been a regular feature in statistical and social science journals since the 1940s. An analysis of JSTOR statistical journals dates the first nonresponse article from 1945 and the Public Opinion Quarterly index's earliest reference is from 1948. The index of Public Opinion Quarterly contains 125 articles on this topic; a full-text search of journals covered in JSTOR finds the following number of articles, by subject area, that included the word 'nonresponse': political science-62, economics-87, sociology-146, and statistics-431
Of course, the complexity of some
of the concepts presented make this edition of the journal a tough read for those
without a survey background. Equations and Greek letters abound. But for the pollsters in the audience - and I
know you're out there - this edition is a must read.
For those thinking about hiring a pollster or survey researcher, I'd
also suggest reviewing the article abstracts and skimming enough of the
articles to form some pertinent questions to the prospective pollster. If nothing else, if you ask, say, what approaches the
pollster takes in "assessing non-response bias" as per Bob Groves' recommendations
, and the pollster asks, "Bob who?" then you know you have
a problem.