If you ever wanted to see a real political focus group from beginning to end (as opposed to the reality TV version that we sometimes see on cable news networks following a candidate debate), thanks to C-SPAN and the Annenberg Public Policy Center, you have a chance. The focus group sponsored by Annenberg and conducted last week in York, Pennsylvania by Democratic pollster Peter D. Hart is available for viewing from the C-SPAN web site, along with Hart's media debriefing. (both links are available from C-SPAN's main page if the foregoing links do not work) And if you would like to watch from the comfort of your sofa, C-SPAN will re-air the entire group tonight at 8:00 p.m. Eastern time.
The Hart/Annenberg focus group was also the subject of a must-read 2,200 word profile this morning by The Washington Post's Robert Kaiser. The success of this particular research tool, as Kaiser points out, "success depends on the skills of the person leading the discussion" and Hart is an especially skilled and experienced moderator. "A bad leader can ruin a focus group," Kaiser writes, "so can one or more ornery participants who try to dominate the proceedings." If you watch Hart carefully, you may notice he guides the group without imposing his own views, and as the evening progresses, gently coaxes the more dominant personalities in the room to hold their opinions for last.
Kaiser is also right that it is often "difficult to understand what is really important in a focus group discussion, and what is just noise." Here are some suggestions:
Be careful about counting. The issue is not so much size, as the difficulty of getting voters to participate in a process that requires in-person participation in a two-hour discussion group. Simply put, focus groups are not random samples, so we cannot use them to arrive at projective estimates of some larger population.
For example, Kaiser observes that "five of the seven Democrats who voted for Clinton in the primary were already comfortable with Obama as their candidate." That is an interesting finding, but it would be wrong to assume that 71% of Clinton supporters in York, Pennsylvania are now supporting Obama. The value is listening to those five individual explain their decision. We can understand the thought process and rationale of those five individuals, even if they do not produce "quantitative" estimates of a larger population of Clinton primary voters.
Who participated? The composition of the group frequently determines the impression that one gets from the discussion. One common philosophy among focus group researchers is to keep the composition as homogenous as possible. In the context of a political campaign, that usually means trying to limit participation to voters that are truly "persuadable" and still uncertain about their choice,
The York group included more "decided" voters than you might find in a typical focus group conducted by one of the campaigns, because in this case Hart and Annenberg wanted to understand how Obama is faring among Clinton's primary supporters. So they allowed primary voters to participate but screened out those who voted for Obama or initially preferred a Republican other than McCain. As a result, the group included quite a few primary voters who are now quite certain about their choice in November. I assume that their collective information level is also a bit higher than a group of truly persuadable voters (like the 23% of Americans described in this Gallup analysis) would be.
Pay attention to what isn't said. Much of Kaiser's piece, as well as the coverage from other journalists who watched from behind the glass, focused appropriately on what the participants said about Obama and McCain. However, I found this passage from Kaiser's article particularly telling:
The discussion, which lasted nearly 140 minutes, demonstrated again and again how little the participants felt they knew about Obama or McCain. "I don't know enough" was the substance of many answers to Hart's queries.
Consider it this way: Near the beginning of the group, Hart asked each participant to write down (and then describe) something that "has had an impact on me" in their thinking about either candidate in this election. Four mention McCain's recent proposal for off-shore oil drilling and three mention his "100 year" statement on Iraq. A few (5) more mention Michelle Obama's "proud of my country" remark and the controversial Reverend Wright (3). But think about the public policy issues that went unmentioned, either at that moment or later in the group. I heard, for example, not a word mentioned at any time about the candidate's positions on the FISA debate or Obama's decision to opt out of public funding for his campaign or a host of other policy issues that the candidates have spoken out on.
What gets said -- and what doesn't -- in this sort of open-ended discussion speaks volumes about the information that ultimately reaches voters in a campaign and what they do with it.
I am certain that our knowledgeable readers, some of whom are survey research professionals, can chime in with their own tips regarding focus groups and their interpretation. Our comments section is open as always. Have at it.