"Do you trust the ARG poll in South Dakota?"
I have been asked that question more than once over the last 24 hours. The South Dakota poll released by the American Research Group yesterday and conducted this past weekend shows Hillary Clinton leading Barack Obama by a 26 point margin (60% to 34%). It is one of only two public polls released on the South Dakota primary. The only other poll, conducted in late March by Dakota Wesleyan University, showed Obama leading by 12 points (46% to 34%).
If the question is about their record during this primary season, we can look at the "Pollster Report Card" for 2008 compiled by SurveyUSA that calculated pollster "error" by comparing the percentage point margin separating the two-two candidates on the final pre-election poll for each pollster in each race to the margin between those two candidates in the actual result. On that basis, ARG shows a median error of 7.0, ranking them 26 on the list of 41 pollsters, and dead last among the nine organizations that polled in 10 or more contests this year.
Back in April, The Wall Street Journal's Carl Bialik took a closer look at ARG after it reported two polls in Pennsylvania that appeared to show the race closing from a 20 point Clinton lead to a 45-45% tie in just one week:
Dick Bennett, president of ARG, acknowledged his firm struggled in early primaries, but told me that its polls in later, big-state primaries have done well, citing California (the final poll showed a Clinton lead of four percentage points; she won by eight), Ohio (ARG had Sen. Clinton up by 14; she won by 10) and Texas (the poll had Sen. Clinton up by three; she won the primary by 3.5 points but appears to have lost the caucuses). According to Mr. Bennett, ARG’s stumbles in states such as Connecticut and South Carolina — where the firm understated Sen. Obama’s support — were due to underestimating the likelihood that first-time voters would go to the polls.
"In the tough ones, we’ve been close,” Mr. Bennett said. “As time has gone on, we’ve gotten much better.”
The most intriguing difference between South Dakota and other recent primaries is that ARG is out all alone with its results. In previous contests -- particularly Iowa, New Hampshire, South Dakota, Ohio, Texas, Pennsylvania and North Carolina -- we had ten or more surveys released over the final weekend. Many pollsters speculate cynically about the potential in such an environment for what one friend of mine calls the "watching the rearview mirror" problem, the idea that some pollsters are watching the results of the competitors and adjusting their own numbers (or weighting targets) so they are not too far off.
The potential irony here is that the big polling story at the beginning of this primary season was the courage of Ann Selzer, pollster for the Des Moines Register, whose final pre-caucus survey showed Barack Obama surging on the basis of a surge in participation by younger and more independent voters. Other pollsters were skeptical and questioned the result. ARG's final poll showed Clinton leading by nine points. Obama ultimately won 38% of the delegates elected that night, while Clinton finished third (with 29%) just behind John Edwards (at 30%). The Iowa entrance poll, arguably a better measure of the initial vote preference, also showed Obama as the first choice of 34% of attendees, followed by 27% ready to support Clinton..
In this case, ARG's John Bennett seems to be comfortable putting out numbers that contradict the conventional wisdom. Give him credit for that, at least. Slate's Christopher Beam contacted Bennett, and Bennett defended his work:
"It’s what the voters told us," he said. "It’s the same process we’ve used in other states." The survey interviewed 600 people who represented the state’s demographics, without the need for weighting or other fancy modeling.
Bennett dismissed the notion that South Dakota will vote like the surrounding states. "Look at New Hampshire compared to Vermont and Massachusetts and New York," he said. "You can’t pick out states like that."
He also pointed out that South Dakota is the oldest of the recent states except for Pennsylvania and West Virginia, and Clinton has performed well among older voters. According to the U.S. Census, 14.2 percent of South Dakota’s population is 65 years or older. In Pennsylvania, the number was 15.2 percent; in West Virginia, 15.3 percent. The Mount Rushmore State also has a closed primary, which tends to favor Clinton—no "independent bonus voters for Obama," Bennett said—and a large proportion of likely Democratic voters are women.
Will the ARG numbers prove accurate tonight? We will soon see.
In other related news:
- The statistical models of FiveThirtyEight's Nate Silver project a 5-point win for Obama in South Dakota and an 18-point Obama margin in Montana. Silver, who was the subject of a nice profile by Carl Bialik this week, also weighs in on the ARG poll -- he's skeptical.
- Brian Schaffner projects the delegate totals with and without the ARG polls. He's also live blogging the evolving delegate count all afternoon and evening.
- I am told that the networks will have exit polls tonight for both South Dakota and Montana, and both will feature supplemental telephone interviews of early voters. So we will see you here later tonight.