04/28/2008 12:05 pm ET Updated 6 days ago

Novak and "The Bradley Effect"

Robert Novak's column last week led with this reference to the Pennsylvania exit poll results:

When Pennsylvania exit polls came out late Tuesday afternoon showing a lead of 3.6 points for Hillary Clinton over Barack Obama, Democratic leaders who desperately wanted her to end her candidacy were not cheered. They were sure that this puny lead overstated Obama's strength, as exit polls nearly always have in diverse states with large urban populations. How is it possible, then, that Clinton, given up for dead by her party's establishment, won Pennsylvania in a 10-point landslide? The answer is the dreaded "Bradley effect."

Prominent Democrats only whisper when they compare Obama's experience, the first African American with a serious chance to be president, with what happened to Los Angeles Mayor Tom Bradley a quarter-century ago. In 1982, exit polls showed Bradley, who was black, ahead in the race for governor of California, but he ultimately lost to Republican George Deukmejian. Pollster John Zogby (who predicted Clinton's double-digit win Tuesday) said what practicing Democrats would not: "I think voters face to face are not willing to say they would oppose an African American candidate."

Unfortunately, Novak confounds two issues, and Zogby's contribution confuses things further. The "Bradley effect" (also called the "Bradley/Wilder effect," the latter based on the 1989 election of Doug Wilder in Virginia by narrower margins than indicated by pre-election polls) pertained less to exit polls but to pre-election telephone surveys. The underlying theory was that white respondents were sometimes unwilling to reveal their preference for the white candidate in a bi-racial contest when they felt some "social discomfort" in doing so. That is, respondents would be less likely to reveal their true preference in a telephone interview if they believed the interviewer supported a different candidate. The most important evidence was an observed race-of-interviewer effect: Support for Doug Wilder in one 1989 survey (pdf) was eight points higher when the interviewer was black than when the interviewer was white.

The problem with extending this idea to the 2008 exit polls is that -- contrary to the apparent assumptions of both Bob Novak and John Zogby -- exit polls do not involve a "face to face" interview. Rather, the exit poll interviewer's task is to randomly select and recruit respondents, hand them a paper questionnaire, a pencil and a clipboard and allow the respondents to privately fill out the questionnaire and deposit it into a large "ballot box."

ballot box.jpg (JPEG Image, 750x123 pixels).png

The more likely explanation for the consistent Obama skew in the exit polls this year is likely less about "voters not willing to say they would oppose an African American candidate," than about the relative youth of the interviewers, and the well established problem that the typically younger exit poll interviewers have in winning cooperation from older respondents. Here is a summary I wrote two years ago about information included in the official, post-election report on the 2004 exit polls:

The [National Election Pool] NEP exit polls depended heavily on younger and college age interviewers. More than a third (36%) were age 18-24 and more than half (52%) were under 35 years of age (p. 43-44). These younger interviewers had a much harder time completing interviews: The completion rate among 18-24 year olds was 50% compared to 61% among those 60 or older. The college age interviewers also reported having a harder time interviewing voters...The percentage of interviewers who said "the voters at your location" were "very cooperative" was 69% among interviewers over 55 but only 27% among those age 18 to 24 -- see p. 44 of the Edison/Mitofsky report.

Given the huge differences by age in both pre-election and exit polls -- Obama wins those under 30 while Clinton dominates among those over 60 -- an age-related selection bias is not surprising. And the issue may not be about simply getting the age mix right in the exit poll. The issue may also be related to the "social discomfort" theories behind the Bradley-Wilder effect.

Respondents may be making judgements about the exit poll interviewers based on their appearance (age, gender and race) that influence whether they agree to participate or avoid the interviewer altogether. Similarly, while exit poll interviewers are supposed to be carefully counting exiting voters and sticking rigidly to instructions that they select every fourth voter (or whatever interval they are instructed to select) anecdotal evidence suggests that those with less experience often deviate from the procedure and "take who they can get." So less experienced, overburdened interviewers are probably making judgments about which respondents (based on their age, gender and race) might be most likely to cooperate.

Again, quoting from my own summary two years ago:

"It's not that younger interviewers aren't good," as Kathy Frankovic puts it (slide #30), "it's that different kinds of voters perceive them differently." Put all the evidence together, we have considerable support for the idea "that Bush voters were predisposed to steer around college-age interviewers" (Lindeman, p. 14) or, put another way, that "when the interviewer has a hard time, they may be tempted to gravitate to people like them" (Frankovic, slide #30).

It is not at all surprising that this same mix of issues -- younger interviewers who have trouble winning cooperation with older respondents and a huge age differential in the results -- produces a consistent skew to Obama in the context of 2008.