07/24/2007 07:12 pm ET | Updated May 25, 2011

Focus Grouping the YouTube Debate

A few more thoughts about the various efforts to determine
who "won" last night's CNN/YouTube debate. In addition to the SurveyUSA panel-back
I blogged
last night, there were (by my count) at least four different focus groups
convened in various locations:

  • CNN reported last night on two "dial groups," one in Manchester, New Hampshire and another in Las Vegas, Nevada, in which selected voters constantly adjusted a dial that rates their reaction to whatever they are watching. (coverage here, here and here).
  • Dial group tester Rich Thau ( did another dial group among 12 likely voters in Bedford New Hampshire. [Update: more on the Thau groups here, via First Read].
  • Fox News had Republican pollster Frank Luntz conduct what appeared to be a more traditional focus group (albeit with a larger than traditional group conducted on live television) in Charleston, South Carolina (transcript, video available from this page). [Update: A reader emails to say that the Luntz group also involved a "dial test"].

Because they are more qualitative, the "results" of these
efforts can sometimes be in the eye of the beholder. Since they involve
small, non-random samples of different kinds of voters, they frequently produce
contradictory results. Some researchers like to use bigger groups (of
40-50 participants), but that approach increases the risk of creating a "group
dynamic" that allows the most opinionated participants to sway others toward
their point of view. We also know little from the various reports linked to
above about the kinds of voters recruited for each group. How many were completely
undecided before the groups? How many were political independents? The answers
to those questions might tell us a lot about the reported outcome in each case.

Not surprisingly, last night's focus groups identified many different "winners." According to coverage from NBC's First Read, "Barack Obama
got the most favorable [response] in terms of the best performance" in the CNN New
Hampshire group
, but their Nevada
showed "Bill Richardson and Hillary Clinton in a tie with six votes
each." On the basis of favorable reaction from his focus group, Frank Luntz
concluded that "Obama will be shown as the winner of tonight's debate."

Meanwhile, depending on which source we turn to, the Rich
Thau group of 12 New Hampshire voters produced good news for John Edwards, Barack Obama and Joe
Biden. Under the headline, "Very Much an Edwards Night," Thau reports
that "a plurality of our group (four of 12) thought Sen. Edwards won the
debate." However, NBC's First Read tells us,
"his survey had Obama doing the best (in terms of improvement from pre-debate
to post-debate; Obama and Edwards tied with the highest post-debate score)." And
Thau's before-and-after "comfort scores" shows
the biggest gains by Joe Biden.

If you're having trouble making sense of all this, you're
not alone.

Also, I am guessing that the audience for the YouTube debate,
while probably larger than usual for CNN, was still a tiny sliver of those who will ultimately vote in Democratic primaries or caucuses. Assuming the ratings followed typical patterns, we
should assume that far more voters saw or read news accounts about the debate
this morning than experienced it in full last night.

In that regard, consider this very sensible observation
made last night by Chris Bowers, on the liberal site Open Left:

I noticed that both on CNN, and in
the progressive blogosphere, several people started trying to determine who
won, and even making declarations that Candidate X had, in fact, clearly won
the debate. I was taken aback. There was a winner? There was even a
competition? What did the winner, well, "win?" New hard-core voters
in the Iowa and New Hampshire primaries? New donors? The
deliverance of a biting comment that will psychological scar one of his or her
opponents so badly that s/he cannot continue on the campaign trail with naught
but a thoroughly shredded sense of self-esteem?

The truth is, there is only one objective way for any candidate to
"win" the debate: if the debate, or the post-debate spin,
cumulatively result in your campaign moving closer to taking the nomination
than you were before the debate. That's it. From this "God's eye"
perspective, it is doubtful that anyone won the debate. The factors that make
the most difference in pushing someone closer to the nomination are, first, new
Iowa supporters and, second, new supporters in
New Hampshire.
Everything else is pretty distant, and I seriously, seriously doubt anyone
picked up much, if anything, in those two categories

That sounds about right.