"Numbers Guy" Carl Bialik devotes his Wall Street Journal column and a companion blog post today to the subject of the automated "interactive voice response" polling that has become such a staple of the current campaign. Both are well worth reading in full.
Bialik managed to interview most of the major players in the political IVR field, and had a reaction from our partner Charles Franklin, summing up our own philosophy regarding the automated polls (that use a recorded voice rather than a live interviewer, and ask respondents to answer questions by pressing keys on their touch-tone phones):
The automated-polling method, says Charles Franklin, professor of political science at the University of Wisconsin and co-developer of the poll-tracking site Pollster.com, "can prove itself through performance or it can fail through poor performance, but we shouldn't rule it out a priori."
The column notes that IVR pollster SurveyUSA ranks second most accurate among all pollsters during the 2008 primaries in the ratings compiled by Nate Silver and that IVR polling was indistinguishable during the primaries in terms of how the final poll compared to the election result:
Their accuracy record in the primaries -- such as it was -- was roughly equivalent to the live-interviewer surveys. Each missed the final margin by an average of about seven points in these races, according to Nate Silver, the Obama supporter who runs the election-math site fivethirtyeight.com.
Franklin did our own compilation of polls conducted during the final week of the 2006 (for a paper presented at the AAPOR conference last year) and reached essentially the same conclusion.
The article also indicates some cracks may be forming in the intense skepticism that the survey research establishment has long held for IVR surveys. Bialik notes that a polling textbook (The Voters Guide to Election Polls ) authored by Paul J. Lavrakas and Michael Traugott, "refers to these surveys as Computerized Response Automated Polls -- insulting acronym intended." But at the end of the column, Lavarakas indicates a willingness to consider the methodology:
Accepting responses by touch tones may have a particular advantage this election, says Mr. Lavrakas, former chief methodologist at Nielsen Media Research, because it may extract more-honest responses from white respondents about their intent to vote for Sen. Obama. "Ultimately the proof is in the pudding, and those firms that use IVR for pre-election polling and do so with an accurate track record should not be dismissed," he says.
Again, this is good stuff. Word reading in full.