Ever since Election Night this past November, I have been ruminating about a story that I've been meaning to post. Its connection to pollsters and poll methodology is indirect, but seems especially relevant right now as we formally turn the page from the incredible election year 2008 to a new administration and its travails in 2009. So before we get back into our usual routine, I want to share it.
In May 2002, I sat down to draft a questionnaire for a new client, a wealthy businessman who was then considering a challenge to an incumbent U.S. Senator two years hence. It was just six years ago, back in the days before my MysteryPollster blog, back when conducting such surveys for Democratic candidates was my full-time occupation. And, as it turned out, this client would be my last "major" race (a designation that, in my mind, falls somewhere between a contest with an uncertain outcome and one with some national significance).
My client's eventual campaign struck some as a microcosm of conventional (and thus flawed) politics. One of the consultants for one of our opponents called our campaign "an exercise in the technology of politics...They do polls and see what people want to hear and then they put ads on the air and run them again and again. There's nothing new about what they're doing." There was truth in this critique, and the ultimate demise of this campaign played a not insignificant role in my evolution from consultant to blogger.
But it was something that happened at the very beginning of that campaign that sticks in my memory now. On that day in May, I started with my usual process when drafting a benchmark "message testing" survey. I took all of my notes from initial discussions with the client, suggestions from other consultants and information gleaned from news clippings and the Internet and synthesized it into a single document listing all of the "messages" -- candidate profiles and arguments about them pro and con -- we hoped to test on the benchmark survey.
I found my "notes for questionnaire" for that project on my hard drive yesterday, still there after moving with me through at least three different computers over the last six years. My first cut consisted of roughly 700 words. About half involved the incumbent Republican, about a third concerned our client. I had also typed a two sentence profile of the potential primary opponent, a former Democratic Senator that had been turned out of office a few years before.
Finally, the document also included the names -- just the names --of three potential candidates that most everyone considered extreme long-shots. The paucity of detail on these three spoke to their status as very long shots in this particular Senate race. Anyone looking over my notes right now, however, would immediately notice one now familiar name among the also-rans:
The survey we fielded a few weeks later showed that just 4% of likely general election voters had a favorable impression of Obama, while 5% rated him unfavorably. Nearly four out of five (79%) had never heard of Obama. He did a little better among likely Democratic primary voters -- 11% favorable, 6% unfavorable -- but still ran far behind, winning just 6% of the vote, in a five-candidate primary matchup that also featured former Senator Carol Moseley Braun. Obama did a little better (rising to 12%) when we omitted Braun, but Obama still ran far behind state Comptroller Dan Hynes (with 34%).
Of course, my client, Chicago businessman Blair Hull, started with even less recognition and support (2% of the vote, to be precise), but his candidacy and its ultimate demise was another story altogether. The consultant who dismissed our campaign as "nothing new" was a truly forward looking guy named David Axelrod. But I digress.
It is still hard to believe that a State Senator who seemed like the longest of long shots for the U.S. Senate so recently is about to be sworn in as the 44th President of the United States. I look back at my notes from six years ago, ponder all that has happened since and just shake my head in wonder.
This story teaches many lessons, of course, but the most relevant to readers of this site is that no poll or statistical model could have predicted Obama's ascent. Yes, we could see from the "internals" of that first survey that Obama had the potential to be a formidable contender in the 2004 Senate primary, especially after both Braun and incumbent Peter Fitzgerald announced that they would not run. But no survey or model in 2002 could have predicted Obama's 52.8% majority in the seven-candidate 2004 Senate primary or the nearly 70% of the vote he won in the general election, much less all that followed in bid for the White House in 2008. I'm certainly a believer in opinion surveys and statistical models, but they have their limits.
The Obama story from 2004 provides no direct corollary to the ongoing arguments about the "electability" of potential Senate candidates in Illinois, New York or elsewhere, but I do see a warning against leaping to conclusions about a candidate's prospects based only on early favorable ratings or horse-race numbers. "Electability" is an appropriate topic whenever a party selects its nominee (or appoints someone to serve out the term of a departing legislator), but quantifying electability through polling, or through predictive models derived from it, is a shaky enterprise at best. Obama's rise -- in the face of early data that led many to question his electoral potential in both 2004 and 2008 -- is a testament to such challenges (see also this pertinent example from September 2007).
This story also says something bigger about the potential for "change" within the American electoral system. Typically the day-to-day business of politics is mundane and static. We fight the same fights over and over, and little seems to change. Politics seems to be mostly about compromise, mostly "the art of the do-able." Yet every once in awhile, usually in the context of a presidential election, some out-of-the-blue candidacy reshapes our perception of what is possible. It shows us that sometimes, if we're lucky, we get to see history being made in the midst of mere electoral politics.
I look back to my own experience in the 2004 Senate election and realize that there was a silver lining in an unsatisfying campaign that amounted to, admittedly, little more than "an exercise in the technology of politics:" It offered me the opportunity to witness (and, yes, literally chart) one of the most meteoric success stories in American political history from the very beginning. And from this perch at Pollster.com, I got the chance to keep that ringside seat, charting and writing about the most exciting campaign of my lifetime.
So I want to thank Pollster.com's readers for coming along with us for the ride, and especially those who have stayed now that things have calmed own a bit. We hope you will stick around and let your friends know as we continue to track public opinion in 2009. You will be seeing a transition in Pollster.com over the next month or so, as we move the 2008 data off the front page and begin to feature charts and data that track the performance of the new administration and the concerns of American voters. But as we do, let's not lose sight of an idea we have tried to stress here from our first post: Using survey data well requires that we understand its limitations.
[Typos fixed and one badly constructed sentence revised].