Despite the long stretch of time between today and the New Hampshire GOP presidential primary, there's an understandable appetite for data about the race as it stands today. Polls today may not be great predictors of the final ballot, but they can give insight into who has (and who lacks) momentum.
Unfortunately, modeling an electorate perfectly is impossible, especially when an election is many months away. Further complicating the matter are harder to reach audiences like young voters. I found myself scanning the most recent Public Policy Polling survey of New Hampshire GOP primary voters to see how young people were responding to the race. What I found raised a number of questions.
First, I found an odd pattern in the responses for the 18-29 group. For any question, the results for the 18-29 banner point broke out as 17%, 33%, 50%, 67%, or 83% of respondents. Normally, when something like that occurs, it is because a cell size is small enough to create that pattern. In this case, you could have as few as 6 total respondents. The survey toplines say that 4% of respondents were 18-29, meaning anywhere from 12 to 15 interviews after weighting.
Second, with only 4% under age 30, it seemed young people were under-represented. In the 2008 primary, 14% of GOP primary voters were 18-29.
I posed these questions on Twitter to PPP (@ppppolls) and they were nice enough to respond. While they did not provide me with the actual cell size of their 18-29 sample, they did reply that they are: "Not seeing a lot of interest from young ppl in GOP primary yet. Suffolk had 18-34 for NH GOP at 10% last week, 18-29 wd be 5-6."
This response intrigued me. I can certainly accept the premise that young people are less interested in the race at this point, but had never thought of this as a reason for them to be under-represented in the sample. Wouldn't it be better to survey a representative sample of who is likely to be there on election day -- including a healthy sample of 18-29 year olds -- and then show that disinterest through their high levels of "undecided," "don't know," "refused," etc? I am unsure if PPP weighted at all by age and, if so, why they chose 4% as the target for the 18-29 age group.
CNN came under fire recently for not having enough young people for a reliable standalone sample, and simply put "n/a" in those cells. PPP in this case gave the results anyhow. Someone reading the PPP New Hampshire crosstabs would come away with the impression that, say, Tim Pawlenty's "base" is young voters (33% of 18-29, but only 6% of the overall sample), or that Mitt Romney has absolutely no support (0%) from those under 30. Those conclusions, if drawn from a small handful of interviews, simply do not stand up on their own.
Kudos to PPP for actually releasing crosstabs, something few pollsters do. I believe that pollsters do want to "get it right" and the tough task of tracking down young people to interview makes it harder and harder to achieve that ideal. In this instance, however, I think the significant under-representation of young voters can harm the quality of the results and give an inaccurate picture of the state of the primary campaign. Pollsters and consumers of polling data both lose out if the 2012 primary narrative is being defined by data that is missing a critical group of voices.