Steve Ansolabehere and I have been working over the past few weeks on a paper we are writing for the AAPOR conference next month. Over the next couple of weeks, I'll share some of our preliminary findings here and I wanted to lead off today by presenting some comparative data we put together from three different surveys with distinct approaches to sampling.
The National Election Study, which has been around since 1948, is a labor- and time-intensive survey that uses a residential sampling approach and conducts face-to-face interviews. The Cooperative Congressional Election Study is an web-based survey conducted by YouGov/Polimetrix. The respondents opt-in to the study and Polimetrix uses a sample matching methodology where they first select a random target sample and then attempt to find a match for each respondent in their pool of opt-in respondents. Finally, Pew's survey work is fairly well known to regular Pollster.com readers. Their efforts to incorporate CPOs into their samples has relied on a dual-frame approach where they randomly select landlines and then cellular lines to build their sample.
So what happens when we compare these surveys, each of which takes a different sampling approach? Each survey I'll present data on here was conducted during the presidential campaign last year. For starters, the chart below compares each survey's estimate of the size of the cell phone only population after sample weights are applied. Note that Pew includes phone status as one of their weighting criteria, but the NES and CCES do not.
CPOs make up 19.7% of the CCES sample after weighting; 17.9% of the Pew sample and 17.3% of the NES sample are CPOs (again, after weights are applied). Thus, the differences are relatively small. Also note that the 95% confidence intervals for each estimate (represented by the darker lines) overlap.
So each of the surveys provides a similar estimate of the size of the CPO population, but what about the composition of that group? The table below compares landline and CPO respondents on a wide array of demographic and socioeconomic factors.
There is a lot of information in the table, but the different surveys are fairly similar across most measures. A few differences do stand out, however. After weighting, the CPO respondents reached by Pew and CCES were less likely to have children, less likely to be married, and less likely to be home-owners than those reached by the NES. Pew's CPOs also tended to have lower incomes and were somewhat more likely to be racial/ethnic minorities. Perhaps NES's face-to-face approach is more likely to pick up CPOs who have "settled down" relative to CCES or Pew.
Speaking of "settling down," one other item deserves attention; that is the information on residential mobility at the bottom of the table. While we weren't able to find this information in any of Pew's surveys conducted in October or November, the data were available for the CCES and NES. Not surprisingly, residential mobility is strongly related to whether one is a CPO or not. Even controlling for age, people who have moved recently are much more likely to have shed their landline and gone with just their cell. In fact, this relationship holds up when you control for all of the other demographic variables in the table. This is something I'll post more about later, but you can probably imagine that there are some significant political consequences arising from the fact that CPOs tend to move around much more frequently.