For pollsters, 2010 was the year in which methodology mattered. Pollsters that invested in more expensive dual samples to reach Americans on both landline and mobile phones should have produced more accurate forecasts of Congressional voting, while those that relied only on traditional landline samples found greater support for Republican candidates. That's the finding of a new report from the Pew Research Center that compared traditional landline samples to surveys conducted over both cell and landline phones.
Pollsters have fretted for nearly a decade about the growing number of Americans in so-called "cell-phone-only" households that are out of reach of traditional landline telephone poll samples. In previous elections, simple weighting techniques largely corrected for the small statistical bias resulting produced by the missing voters, but the missing slice is now estimated at more than quarter of U.S. households, big enough to affect polling accuracy.
The new Pew Research analysis confirms findings from an early October report, released in early October, which found the Republican statistical skew evident in landline-only samples growing since 2008. Subsequent surveys conducted by the Pew Center in October yielded the same pattern:
Across three Pew Research polls conducted in fall 2010 -conducted among 5,216 likely voters, including 1,712 interviewed on cell phones -- the GOP held a lead that was on average 5.1 percentage points larger in the landline sample than in the combined landline and cell phone sample. In six polls conducted in the fall of 2008, Barack Obama's lead over John McCain was on average 2.4 percentage points smaller in the landline samples than in the combined samples.
In Pew Research's final pre-election poll in 2010, the landline sample of likely voters found Republican candidates ahead 51%-39%, a 12-point lead. In the sample that combined landline and cell phone interviews, the Republican lead was 48%-42%, a six- point advantage. The national vote for House candidates is not yet final; currently, Republicans lead by approximately a seven-point margin.
Most remarkable about the new Pew Research findings is how closely they parallel the differences among the other public polls conducted at the national level this fall. As shown in the table below, the seven organizations that interviewed Americans over landline and cell phones gave the Republicans a roughly six-percentage point advantage on the national House votes in their final-week surveys. The four pollsters that interviewed via landline only gave Republicans a lead averaging more than 10 percentage points.
The most recent tallies of U.S. House vote preference from the Associated Press give Republican candidates a nearly seven percentage point lead nationwide (52.0% to 45.3% with 2.7% of the vote cast for other candidates), although those totals may still exclude some not yet counted early and provisional ballots. They also exclude a dozen congressional districts where no votes were cast because incumbents were unopposed.
The Pew Research report also finds that the pro-Republican skew is not just a matter of those missing because they live in "cell phone only" households. There is also the issue of how pollsters reach the nearly two-thirds of Americans who use both landline and cell phones. The Pew Research surveys allowed for a comparison of these "dual users" reached either by cell or by landline phone:
Dual users reached on their cell phone differ demographically and attitudinally from those reached on their landline phone. They are younger, more likely to be black or Hispanic, less likely to be college graduates, less conservative and more Democratic in their vote preference than dual users reached by landline.
Among dual users reached by landline, Republicans had a 12-point advantage among likely voters. But the GOP lead was only five points among dual users reached by cell phone. Among cell phone only voters, there was no Republican lead (a nominal 47%-44% Democratic edge).
Thus, future accuracy in polling may not be a matter of merely calling "cell phone only" voters via cell phone. Pollsters may need to conduct parallel surveys of "dual users" via cell and landline phone. The Pew Research report found merely "adding just the cell-only respondents to the landline samples reduced the GOP advantage," from more than 12 to roughly 10 points, "but not as much as including all of the interviews from the cell phone sample," which reduced the Republican lead to the more accurate 7.6 percent margin.
Follow Mark Blumenthal and HuffPost Pollster on Twitter