Will Party Weighting Account for the Cell Phone Only Problem?

10/01/2008 12:38 am ET | Updated May 25, 2011

The key takeaway from the recent Pew Report on cell phone only users was not that cell phone only respondents are different, but that even weighting landline only survey data doesn't fully account for excluding cell phone only users. Typically, a survey may be weighted for factors such as age, race, gender, education, and region. This allows pollsters to take a particular sample, and adjust it to look more like what they think the population they are interested in actually looks like. Pew found that even if you weighted a landline sample for all of these factors, that sample still provided results that were 2%-3% less favorable for Obama than one that included cell phone only users.

What exactly makes cell phone only respondents different from those with a landline? If it is simply the case that cell phone only respondents are more likely to be Democrats than those with landlines, then it should be simple enough to correct for not calling cell phones by weighting a sample by party identification. From my reading of the Pew report, they did not examine whether applying a party weight would have accounted for the exclusion of cell phone only respondents. A party weight is something that some pollsters (like Rasmussen) apply, but others do not. However, based on some recent analysis I have conducted using the 2006 Cooperative Congressional Election Study, even weighting by party is not likely to fully account for the differences between cell phone only respondents and those with landlines. There are two reasons for this.

First, it is true that cell phone only respondents are more Democratic than landline respondents. But this relationship is a little more complex than it first seems. In the 2006 survey, cell phone only respondents were just 4% more Democratic than landline respondents and they were 9% less Republican when asked a standard party identification question. However, once you factor in independents who lean towards the Democratic or Republican Party, you find that cell phone only respondents are 10% more Democratic and 12% less Republican. Thus, the party differences are larger when you factor in leaners, a pattern that results because cell phone only respondents are more likely to initially call themselves independents even though they lean Democratic. To fully capture party differences among cell phone only respondents, one would need to factor in leaners.


Second, just looking at party affiliation masks the fact that cell phone only respondents are actually quite a bit more liberal than those with a landline. 35% of cell phone only respondents classified themselves as liberals compared to just 23% of those with a landline. These ideological differences are not completely accounted for by party either. From the table below, you can see that cell phone only Democrats are 10% more liberal than those with landlines. Democratic leaners in the cell phone only sample are 15% more likely to classify themselves as liberal. And even those cell phone only independents who did not express a lean to either party were more likely to be liberal compared to their landline counterparts. Given that cell phone only Democrats and Democratic leaners are more liberal than those with landlines, they should be less likely to defect and vote Republican than landline Democrats.


Thus, this analysis suggests that differences between cell phone only users and those with landlines cannot simply be accounted for by partisanship. In fact, even when I used multivariate models controlling for a wide range of demographic and political factors (party, age, race, gender, income, education, and even religion), cell phone only respondents were still substantially more liberal than those with landlines. Cell phone only respondents are ideologically distinct in ways that cannot be accounted for by party identification or all the other standard demographic factors that pollsters may use to weight samples.