[Editor's Note: We are pleased to add yet another contributor to the Pollster.com lineup. Kristen Soltis is currently the Director of Policy Research for The Winston Group, a Republican affiliated public opinion research and strategic consulting firm in Washington, D.C. Welcome Kristen!]
The debate over party ID and whether or not weighting for party ID is appropriate has raged on for years, with a very thorough treatment by Mark Blumenthal and others that raises good questions about whether or not party ID is stable at the individual level. Recent media polls with wide ranging spreads between Republicans and Democrats make it all the more appropriate to bring this debate back.
Those on the side favoring weighting say that it is important to compare "apples to apples", to see if more people actually are voting for Obama than last month, or if we just happened to get a sample more favorable to him. On the other side, you have folks who view partisan identification as a question response, not a demographic group, and view weighting by party as methodologically unsound.
Though it's controversial, I believe that weighting for party ID is appropriate if done in a manner consistent with historical norms. I fall into the camp that believes party ID is far more static - that voters can change their preferences and the intensity of their partisanship often, but do not as frequently take the step of giving themselves a new party with which to identify. To me, party ID falls somewhere in between "demographic fact" and "variable question response". Preventing wildly fluctuating data outside historical norms provides a better picture of what real movement is occurring in the electorate on questions like the ballot test.
On Election Day, the partisan makeup of the electorate is rarely dramatically different from the election four years prior, and the exit polls from the last twenty years corroborate this. The National Election Study at the University of Michigan back in the 1960s showed party ID was stable at the individual level, but some have dismissed this as an example that works today. So let's take a look at more modern day politics, with a time frame of last twenty years (presidential elections since 1988). Washingtonpost.com has a great, simple table of this exit poll data.
In 1988, Democrats had a three-point party ID advantage over Republicans (38-35). In 1992, Democrats still had a three-point party ID advantage over Republicans (38-35). In 1996, that advantage increased to four - a shift of one point (39-35). In 2000, Democrats were steady, up by four (39-35), and in 2004 they dropped to even (37-37).
During presidential years, over the last five presidential elections, the biggest party ID gap was four points, and the greatest swing was four points as well.
Arguments can certainly be made that in this environment, Democrats should be expected to have a huge partisan shift in their favor. But note that in 2006, when Democrats clearly found enormous success at the ballot box, that the advantage in party ID was only three points (38-35). Polls leading up to the election showed party ID gaps as big as eleven points (Newsweek's poll on Oct 5-6, 2006), rarely showing party ID gaps of less than +5 for the Democrats.
On Election Day, as measured by the exit polls, the party ID divide was just three points.
Just because people are voting Democratic doesn't mean they are becoming Democrats.
Truth be told, the decision to use weights for party ID has everything to do with whether or not a pollster views party ID as a "response" or a "demographic", and when it is a fairly stable characteristic of the electorate, I feel comfortable placing it on the spectrum closer to "demographic". It's not perfect, to be sure, but I'd rather compare surveys month to month and observe movement by comparing apples to apples.
However, whether or not weighting is used, the partisan makeup of a poll must factor into the understanding of whether the poll is presenting a realistic piece of information. I certainly don't believe all polls must weight for party ID in order to be useful. But regardless of whether the party ID is organic or weighted, it should still look reasonable.
So let's take a current example that I have trouble with. As "bambi" noted in the comments (taking quite a bit of heat, and with some calculations that I do disagree with) just this morning about the most recent CBS poll, after weighting for demographics, the difference between Republicans and Democrats nearly doubles. While the unweighted sample has 317 Republicans and 381 Democrats (out of 1034 adults), the weighted sample has 284 Republicans and 406 Democrats. This changes the spread from a 6 point spread (31-37) to a 12 point spread (27-39).
Truth be told, if a poll shows a six-point party ID spread, I wouldn't immediately dismiss it. Furthermore, the CBS poll is of adults, not registered or likely voters, so that gives it freedom in my opinion to veer a bit outside the norms. I'm not dogmatically tied to historical precedent, though I think it's very instructive in determining what is "reasonable".
But a twelve point spread? Whether this is a blip or what consistently turns up in the numbers, I have incredible difficulty believing that a margin of that magnitude is an accurate reflection of the electorate. A six-point lead is within the realm of possibility given a really great year for Democrats. But a twelve-point spread is simply outside the bounds of history, given that in twenty years of political change and history, the greatest margin has been four.