12/18/2007 08:56 pm ET Updated May 25, 2011

"Electability" Surge or Order Effect?

Did the order of questions help create the result that led this morning's story by Susan Page about the new USA Today/Gallup poll? That possibility is worth considering, at least. Here is the headline and relevant text :

Poll: Electability key among Democrats

WASHINGTON — Democratic voters increasingly are focused on nominating the most electable presidential candidate, a USA TODAY/Gallup Poll finds, and Illinois Sen. Barack Obama fares better than New York Sen. Hillary Rodham Clinton against prospective Republican rivals...


In a shift, Democratic voters are almost evenly divided between those who want a nominee who agrees with them on almost all issues and those who want one with the best chance of beating the Republican. Last month, they preferred an ideological match by 3-2.

The full text of the "electability" question is as follows:

5. (Asked of Democrats and independents who lean to the Democratic Party) Which type of candidate would you prefer to see the Democrats nominate for president in 2008; a candidate who agrees with you on almost all of the issues you care about but does not have the best chance of beating the Republican, (or) a candidate who has the best chance of beating the Republican, but who does not agree with you on almost all of the issues you care about?

In November, Democrats preferred the better-on-the-issues candidate by a 58% to 36% margin, but that margin faded considerably (49% to 45%) on the survey conducted this week.

One concern I have with this sort of measure is that is forces a choice that most real voters have not considered (or at least, have not considered in those terms). Consider this question from the just released Diageo/Hotline survey on which "winning the general election" ranks last:

I'm going to read you a list of traits people might look for in a presidential candidate. Please tell me which candidate for president to have? [Result among Democrats/Democratic leaners]

50% Will lead the country in a new direction

22% Has the experience necessary to be president

17% Is an inspirational leader

4% Has the best chance of winning the general election

11% (All of the above/none/not sure)

At very least, the Diageo/Hotline results provide different perspective on just how important "electability" is to Democrats nationally. However, they do not explain why preference for the electable candidate went up by nine points in a month on the USA Today/Gallup question. For that we need to consider question order.

In both the November and December Gallup surveys, the issues-vs-electability question was preceded by presidential candidate favorable ratings and (for Democrats) a trial-heat question featuring the entire Democratic field and another featuring just Clinton and Obama. In November, the issues-vs-electability question immediately followed the Obama-Clinton matchup, but in the survey this week, a new question came in between:

4. (Asked of Democrats and independents who lean to the Democratic Party) If you had to choose, would you rather see the Democrats nominate someone for president in 2008 who is very conservative, conservative, moderate, liberal, (or) very liberal?

30% liberal/very liberal

48% moderate

19% conservative/very conservative

Could a question tossing out the terms "liberal" and "conservative" -- particularly given that two-thirds of Democrats say they prefer a "moderate" or "conservative" candidate -- have primed the importance of selecting a candidate with "the best chance of beating the Republican" enough to increase preference for such a candidate? I submit that it may have done just that. The fact that preference for a "liberal" candidate is six points lower than the percentage of Democrats that consider themselves "liberal" provides some support for that argument.

As always, we cannot know for certain without a "split sample" experiment that tests both versions (with and without the ideological preference preceding the "issues-vs-electability" question) on the same survey. However, we ought to consider the possibility that the apparent surge in desire for an electable candidate among Democrats is just a question order effect.