This certainly seems like a banner week for blogging about pollster Scott Rasmussen, as I count at least three entry-worthy topics on the automated polltaker: (1) the flurry of commentary surrounding the piece by Politico's Alex Isenstadt on attacks from the left on Rasmussen's credibility (2) reporting Monday by the liberal blog Think Progress showing that Rasmussen was paid $140,500 by the 2004 Bush campaign for survey research and the good question that raises about whether sites like ours should label Rasmussen as "[R]" for Republican (3) yesterday's new Rasmussen survey of likely voters in this month's special election in Massachusetts.
As commenting on all three at once exceeds both my time and mental bandwidth, I'm going to start with the third and most timely topic, but I will come back to the other two later this week.
Rasmussen's Massachusetts survey, consisting of 500 automated interviews of Massachusetts likely voters conducted in just one day (Monday), shows Democrat Martha Coakley leading Republican Scott Brown by just nine percentage points (50% to 41%). That's a surprisingly narrow lead in a heavily Democratic state that Barack Obama carried by more than 26 percentage points (61% to 36%). Even in 1994, a banner year for Republicans, Ted Kennedy defeated Mitt Romney in an unusually competitive reelection contest by 17 points (58% to 41%).
Nate Silver looked at the Rasmussen results by party and extrapolated that the survey consisted of 52% Democrats, 21% Republicans and 27% in the independent/other category:
Although there are lots of different ways to ask about party identification, typically that's not what we see in elections in the Bay State, as the number of independents is usually much higher (43 percent of Massachusetts voters were independent/other in 2008, and 51 percent are registered as independents). They're also showing an electorate that is 39 percent liberal, 34 percent conservative, and 27 percent moderate; that compares to 2008 exit poll demographics of 31 percent liberal, 19 percent conservative, and 49 percent moderate.
So Rasmussen's theory on this election, basically, is that the people in the middle won't bother to show up; there are many fewer independents and many fewer moderates in their sample than you usually get in Massachusetts. Instead, it will be a race between the bases.
My first reaction is that while there are indeed different ways to ask about party identification (and even more ways to ask about self-reported ideology), it's a bad idea to compare official party registration statistics (that tally how many voters check the box for Democrat or Republican when they register to vote) with survey questions about party identification (typically: "when it comes to politics, do you think of yourself as a Democrat, a Republican or an independent?"). Depending on the state, the two measures can produce very different sets of numbers.
Back in October, Monmouth University pollster Patrick Murray explained why officially "unaffiliated" voters in New Jersey are very different from the "independents" identified by pollsters. I suspected that Massachusetts, with its very high percentage of non-partisan registrants, might produce similar differences, so I emailed Andrew Smith, director of the University of New Hampshire Survey Center who frequently conducts surveys of Massachusetts for the Boston Globe. Here is his take:
[It's] important to point out that a high percentage of the registered Independents in MA (they're actually called unenrolled) are really either Democrats (36%) or Republicans (34%) when you look at PARTY ID. (We use the Univ of Michigan question and I recode leaners into the partisan buckets). Calling them "Independents" makes it look like there is a large pool of free-thinkers out there up for grabs, which is simply not the case ... not in MA, not in NH (a regular media story during the NH Primary is about the large number of Independents up for grabs, a story which sounds good, but has no basis in fact!), not anywhere!
Those who are registered Unenrolled in MA are less interested in elections and less likely to vote than are registered Republicans or Democrats. This phenomenon is consistent across the US (see "The American Voter Revisited" for the most recent in a long line of studies making this point). It's my sense that the 2010 MA special election will have low turnout and the percentage of voters who are registered as either Democrat or Republican will be higher than the percentage registered as such among all MA adults.
Let me explain that a little more slowly. The Boston Globe/UNH poll routinely asks respondents about both their party registration and their party identification. Early in their interview they ask: "Are you registered to vote as a Democrat, Independent, Republican or something else?" On their September survey, according to data Smith provided, 36% of all registrants said they were registered as Democrats, 14% said they were Republicans and the rest (50%) reported they were unenrolled.
Then at the end of their survey they ask: "GENERALLY SPEAKING, do you usually think of yourself as a Republican, a Democrat, an Independent or what?" To those who initially identify as independent, identify with another party or offer no preference they ask a follow-up: "Do you think of yourself as closer to the Republican or to the Democratic party?"
When Smith combined the initial identifiers with leaners in September, he found 50% were Democrats, 32% were Republicans and only 19% remained independent or without a preference. And a cross-tabulation of the UNH party registration and party identification questions shows that more than two thirds of the unenrolled voters identify or lean to either the Democratic (36%) or Republican (34%) parties.
So unaffiliated/independent was 50% on the party registration question, but only 19% on party identification (with leaners allocated). So again, it is a mistake to expect the party identification results produced by a poll to match the party registration statistics produced by the Secretary of State (and an even bigger mistake to weight results of a poll to match those statistics, but that's a topic for another day).
But wait, the party identification results I cited are for adults. Does the Democratic advantage narrow among "likely voters" in Massachusetts?
Yes, although the results that Smith provided from the September Boston Globe/UNH poll did not ask about the special election. They did ask, however, about interest and likelihood to vote in the general election for governor in November 2010, and thanks for Andrew Smith, I can provide those tabulations below:
As the table shows, if you narrow the survey based on interest in the gubernatorial election, the Democratic advantage narrows considerably, from 17 percentage points among all adults (49% to 32%) to just three percentage points among the 35% of adults that say they are extremely interested (45% to 42%). On the other hand, when the UNH pollsters asked voters if they were likely to vote in the November 2010 general election, the Democratic advantage actually grows slightly, to 19 points (51% to 32%). Which of these, or what combination, might provide the best "model" of a true likely voter? There's no obvious answer -- welcome to the highly varied "art" of likely voter modeling.
But wait...the point of all of this is how these numbers compare to extrapolated party numbers produced by Nate Silver for the Rasmussen poll, and at first blush, the Globe/UNH numbers are not that far off. If anything, Rasmussen's party results (52% Democrat to 27% Republican) are more favorable to the Democrats than the Globe/UNH numbers from September.
I had even considered including the Rasmussen numbers in the table above, but decided against because the comparison is a bit misleading. The problem is that Rasmussen asks a very different partisanship question:
If you are a Republican, press 1. If a Democrat, press 2. If you belong to some other political party, press 3. If you are independent, press 4. If you are not sure, press 5.
Does this question ask about party identification or registration? Given the absence of the "do you consider yourself" clause and the use of "belong to, a respondent might interpret it either way. And I'm assuming that since Rasmussen uses an automated method, the respondent can interrupt the question at any time to choose a selection, something I suspect they tend to do more readily toward the end of the interview (especially if they are feeling impatient to get off the phone).
Also, notice that unlike the Globe/UNH question, Rasmussen does not include a follow-up to press independents on how they lean. Yet if you click on a Rasmussen result on our national party ID chart, you will see that when asked of national adult samples, Rasmussen results tend to produce more partisans (about 12 percentage points more, on average) than other pollsters (roughly 9 points higher on Republicans and 3 points higher on Democrats).
Why is it different? Does the combination of a different question and mode effectively push some independents to say how they lean (with a bigger push toward the GOP)? Or do Rasmussen's sampling and calling procedures yield an adult sample that skews more partisan and Republican, even before they apply their likely voter screen and party weighting? You can make a case for either argument, but I don't have conclusive evidence to resolve this puzzle.
Also, Rasmussen typically weights their statewide pre-election samples by party to targets derived in a somewhat fuzzy process. How did they determine their party weighting targets for the Massachusetts survey? How much did the party weighting alter the results (as compared to weighting on demographics alone)? What percentage of Massachusetts adults passed the screen and qualified as likely voters?
And most important, why aren't answers to these questions disclosed on a routine basis on RasmussenReports.com? Keep in mind that Nate Silver had to extrapolate his estimate of Rasmussen's partisan balance, and even that came from crosstabs available to subscribers only.
I'll have more to say later about the questions of bias, intentional and otherwise, that have been swirling around Rasmussen this week. But until pollsters like Rasmussen start disclosing more about the numbers they produce, it is hard to do much more than speculate about whether poll like the one he did in Massachusetts are as representative as they should be. Is this new poll "horribly, terribly wrong?" With so little information to go on, it's hard to say.
Update: Harry Enten (aka "poughies"**), the Dartmouth student who wrote a guest contribution a month ago on modeling gay marriage referenda, takes issue with my conclusion with an intriguing comment below that argues that the Rasmussen poll "has too many Republicans and not enough independents." He reaches this conclusion by comparing the relationship between results for party ID and actual registration in a previous Massachusetts congressional district race polled by SurveyUSA (which asks a more traditional party identification question). It's worth reading.
**And yes, he gave me permission to reveal his identity.