Pollster.com reader SH wrote to ask what I think about the "SurveyUSA match ups on the front page of their website?" SH points out that,
McCain and Giuliani are both unbeatable in the Electoral College. They show Giuliani beating Clinton by 170 electoral votes, for example. I'm just curious if you could speak to the veracity of their numbers and methodology.
For those who haven't clicked the advertisement that has appeared occasionally at the top of the Pollster.com page (and thank you, if you have), the automated pollster SurveyUSA has recently replaced their front page with a promotion for subscription-only access to a series of head-to-head general election presidential match-ups. Their site claims to offer tests of 60 different match-ups, each with results in all 50 states plus the District of Columbia.
The list of match-ups includes the obvious potential candidates, the long-shots and some that range from fanciful to purely hypothetical (including Oprah Winfrey, Bill O'Reilly, Ronald Reagan and Franklin D. Roosevelt). Non-subscribers can pick one match-up and see the map color to show the Electoral College winner (presumably the nominal leader is the presumed "winner" regardless of sampling error). Access to actual data and cross-tabs requires a paid subscription.
Dig a little deeper on the SurveyUSA site and you will find their rationale for this ongoing 50-state project:
For the past 70 years, public opinion researchers in America, trying to figure out who will be the next president, have conducted polls of "Americans," and asked them if they would vote for Candidate A or Candidate B. The pollsters then report the results of these nationwide polls and tell America what America thinks.
But ... there's a problem with this.
America does not have national elections. Presidential elections are won state-by-state. And the only votes that matter, in a presidential election, are Electoral Votes.
True enough, but with the election still nearly two years away, do all those match-ups really allow us to "handicap the 2008 field with breathtaking, and sometimes unexpected, clarity," at "a level of precision never before contemplated," as the SurveyUSA site claims? Put me down as skeptical.
Now don't get me wrong. The simultaneous 50-state surveys that SurveyUSA has conducted since May 2005 have made a unique and valuable contribution to our ability to follow the job-approval ratings of President Bush and the governors and Senators of all 50 states. Those surveys deserve many of the adjectives used in the prior paragraph. And as the presidential nomination process gets underway in earnest and the public begins to learn more about the likely nominees, a small subset of these 50-state presidential pairings will get progressively more useful.
But right now? Most of the real potential candidates currently register somewhere between vaguely familiar and totally unknown to most Americans. When asked to choose between a candidate they know and a candidate they have never heard of, a plurality will usually choose the name they know. The hardest of hard core partisans will choose on the basis of party affiliation, while those who want to know more will opt -- appropriately -- for the "undecided" response. So at the moment, most of the head-to-head match-ups reflect the state's partisanship combined with each candidate's current the level of recognition.
A favorable rating (or something comparable) provides a more useful measure, for the moment, in probing more directly about the level of awareness and popularity of each potential candidate. And even then we need to be aware that those ratings are virtually certain to change for the likely nominee, as well as for those that become active candidates.
Consider, for example, the ratings of John Kerry as measured by the Gallup Organization on adult samples prior to the 2004 campaign. At this time in 2002, Kerry received a rating of 31% favorable, 13% unfavorable, while more than half (57%) had either never heard of Kerry or could offer no opinion. A little over a year later, just before the Iowa Caucuses in January 2003, Kerry's national favorable rating remained the same (31%) but his unfavorable rating had grown to 32%. Even then, more than a third of Americans (37%) could not rate him. After the New Hampshire primary, his favorable rating shot up to its peak of 60%, and while most Americans could rate him, roughly one in ten (12%) could not.
By the time the party conventions were complete, all but roughly 4% of Americans could rate Kerry. His favorable and unfavorable ratings varied only slightly (from 50% to 52% favorable, 43% to 44% unfavorable) on Gallup's surveys of adults conducted in September and October 2004.
The point is that two years out, most Americans had little awareness of John Kerry, while virtually all knew him by the fall of 2004. So theoretical match-ups between Kerry and George Bush tested at this time in 2002 would have been of little value in handicapping the race to come.
What about the potential candidates that are already better known, such as Hillary Clinton, John McCain or Rudy Giuliani? As recent surveys by Gallup and CNN show (via The Polling Report), only Clinton currently has the sort of recognition that Kerry achieved prior to the 2004 elections. Roughly a quarter of the American public cannot rate even well known candidates like McCain and Giuliani.
So match-up tests between the better known candidates are more valuable, but still potentially misleading. Two things are certain with regard to the 2008 presidential election. Each party will choose a nominee, and by Labor Day 2008 those individuals will be far better known, with public profiles that add up to something very different from the way Americans perceive them today.
A currently lesser known candidate, if nominated, will go from unknown to familiar. Should a better known name like McCain or Giuliani win the Republican nomination, more voters will know them and all voters will know more about them. Much the same can be said for Hillary Clinton. If she wins the Democratic nomination, do we really think voters will perceive her the same way in the fall of 2008 as they do now?