Let me start with the "my bad" portion of this entry: Three weeks ago, our colleague David Moore sent me an early draft of the "Dubious Polls Awards" commentary he co-authored with George Bishop. Moore asked for my comments, but in an oversight that speaks to my own poor management of an overflowing email inbox, I set the message aside without reading the attachment and soon forgot about it. He ultimately posted a summary here earlier today, with the more detailed version posted on stinkyjournalism.org. Had I read the draft, I would have given David feedback consistent with what follows. I apologize to David and our readers for that oversight, but I want to take this opportunity to air the issue publicly and allow readers to react and comment.
I make no apologies, on the other hand, for giving David Moore (and by extension, George Bishop), the opportunity to blog here at Pollster. As I noted earlier this week, David brings to this endeavor a long career in the field of survey research, as an author, an academic and a former managing editor of the Gallup Poll. George Bishop is one of the most respected academic survey researchers, and though his perspective is sometimes at odds with others in the field, his work is something any serious pollster should know (particularly his book, The Illusion of Public Opinion ). If Moore and Bishop are willing to to act as provocateurs and criticize the most respected voices in the field, fine. They have the expertise to do so with authority, and constructive criticism has always been part of our mission.
I also believe that blogging works best when edited least. Holding back posts for review and revision kills the spontaneity and give-and-take that make blogging work. As Andrew Sullivan has written, readers are his best editors. "E-mail seemed to unleash their inner beast. They were more brutal than any editor, more persnickety than any copy editor."
The only rule I have tried to set for contributors here at Pollster is to follow my tone: Avoid name calling and gratuitous snark and, above all, be fair.
My problem with the Dubious Polls summary -- and the feedback I should have given Moore and Bishop -- is that it offers far too much snark and name calling with just a smattering of the smart context that they are well equipped to provide (and do a better job providing in the longer version on stinkyjournalism.org). It is also, in places, less than fair.
Consider, for example, their "top award, earning five crossed fingers:"
[It goes to] all the major media polls for their prediction of Giuliani as the early Republican frontrunner. Collectively this group, beginning more than one year prior to the first statewide electoral contest in Iowa, relentlessly, and without regard for any semblance of political reality, portrayed Rudy Giuliani as the dominant Republican candidate in a fictitious national primary.
It is certainly true that most public pollsters reported results showing Giuliani leading, consistently, throughout most of 2007, on questions that asked Republican identifiers nationally to state their preference for the Republican nomination. And it is also true that far too many journalists and pundits (and some pollsters) looked at these early results, showing Giuliani with the support of just 30% of Republicans nationally, and wrongly assumed or predicted that the former New York mayor had some sort of lock on the Republican nomination.
If Moore and Bishop had argued in their summary that we should have paid more attention to polls in New Hampshire and Iowa than nationally,or that pollsters should have done more to caution poll consumers against reading too much into those early Giuliani leads, I would agree (and did, here and here, back in August 2007). I also agree that any "predictions" of a Giuliani triumph based on those 2007 horse race polls alone ignored many political realities, including the fact that we do not hold a single-day, national presidential primary.
But is it fair to characterize as a "prediction" every horse race result released by the ten organizations Moore and Bishop list? Is it fair to use the phrase "crossed fingers" -- words that imply deliberate dishonesty -- to depict the release of those results? It feels unfair to me.
Obviously, this site is not my exclusive domain. Our goal for Pollster.com is to present a wide variety of poll and survey related content from many different authors, and we do not expect every front page contributor to agree or speak with one voice. On the question of tone, however, I want to hear from you. Please read over the Dubious Polls piece here, and the longer version on stinkyjournalism.org. What advice would you offer -- to David Moore or to me -- for future contributions?
Please feel free to comment below or email me directly. I will try to post excerpts from email in a future post (please stipulate if you prefer that your comments to remain totally off the record).