Yesterday I speculated about whether we might see the return of largely abandoned "incumbent rule" which some of us used to interpret pre-election polling in the 1980s and 1990s but that largely disappeared over the last decade. And then last night, in Arkansas and Pennsylvania, we saw exactly the pattern that the old "rule" would have forecast: The challengers in both states performed much better than the final pre-election polls would have suggested.
Having re-opened this can of worms, you might expect me to crow about a newly revived "incumbent rule," but having thought on it further, I want to offer some cautions. We ought not make too much of the results of two somewhat unique primary elections. The current political environment creates huge opportunities for little-known political outsiders, to be sure, but "rules" of interpretation require far more empirical support than the examples from last night.
Let's review how the final polls compared to the outcome, starting with Pennsylvania:
Our final trend estimate based on all the polls (not just the four from the final week listed above) shows the two candidates essentially tied (43.1% for Sestak to 42.5% for Specter), but Sestak won by 8 points (54.0% to 46.0%). Some will read these results as representing a roughly three-to-one "break" of the undecided vote to Sestak and thus fitting the theory behind the incumbent rule: Undecideds voters are really undecided about the incumbent and thus end up deciding to support the challenger at the last minute. Some of that may have been true, but I can think of a number of reasons to be cautious about such a conclusion:
First and foremost, the sum of all polling in this race showed a huge trend to Sestak that kicked into high gear after he started running television advertising on April 20. That trend ends on our chart with a virtual tie, but we should remember that only two polls continued calling through the weekend, and those two both wrapped up on Saturday. All four in the final week had field periods of 3-5 days, so the final polling snapshot had an epicenter of roughly Thursday.
Some looked at the results of the daily rolling-average tracking conducted by Muhlenberg University and saw a leveling off of the trend, but as I argued last week, it is all too easy to make too much of the sampling noise inherent in this sort of tracking. Muhlenberg tracked for 18 days. If we compare their first eight days (April 20-May 6) to their last eight (May 9-May 16) by averaging non-overlapping releases, they show Sestak closing a six point deficit (41% to 47%) to a dead heat (45% to 45%). At the same time, they show Sestak's personal rating jumping nine points, from 44% favorable-12% unfavorable to 53%-14%, but with nearly a third of likely voters (34%) still unable to rate him during the final week.
Second, we have good reason to assume that both trends -- Sestak gaining recognition and vote along with it -- continued and even accelerated over the last few days of the campaign. Local television stations and newspapers that had covered the race only sporadically all did Senate race stories over the last few days. Our trend line ends on Saturday, May 16.
Third, this race is one instance where our "more sensitive" trend estimate (available through the "smoothing" tool on the chart) provided a more appropriate summary of the data. You can make your own judgement by using the embedded chart below, which I have set up to show just Sestak's vote using the more sensitive setting. Try clicking through to the fully interactive version and toggling both the sensitivity setting (Tools/Smoothing) for both Sestak and Specter (Tools/Choices). The more sensitive setting showed Sestak with a 3.3 percentage point lead (44.7% to 44.1%) as of Saturday. That estimate is closer to the actual result.
Fourth, the data I obtained yesterday about the final week undecided voters from both the Quinnipiac and Muhlenberg polls did not suggest a hidden anti-Specter vote. Fewer than one voter in five of those who were still "undecided" had formed a negative opinion of Specter, although roughly half were neutral or expressed no opinion of Specter. If Sestak continued to gain support it probably came both from totally undecided voters and others who would have expressed a weak preference for Specter until they learned more about Sestak over the final weekend
Fifth, and probably most important, the problems of sampling and identifying likely voters confounds our ability to determine how undecideds "break," and both tasks are much, much harder in primary elections. Consider that only 12% of Pennsylvania's registered voters, and only 24% of its registered Democrats, cast a ballot in the Senate primary. So pollsters have a huge challenge identifying voters even when sampling from registered voter lists (never mind the related obstacles of covering those with missing phone numbers or that live in cell-phone only households).
Thus, it would not surprise me at all if many polls sampled the likely electorate too broadly -- including too many non-primary voters who might find a Republican-turned-Democrat like Specter more appealing. If so, the lessons learned yesterday are not likely to apply to general election polling in the fall.
Now let's turn to Arkansas:
At first blush, the Arkansas results seem like an even better example of the incumbent rule in action. Sen. Blanche Lincoln won virtually the same percentage of the vote has she received on our final trend estimate (both round to 45%), while challengers Bill Halter and D.C. Morrison picked up support as compared to the final polling.
The Arkansas polling yielded no clear trends comparable to Pennsylvania, but these results present other challenges. Only two pollsters (Daily Kos/Research 2000 and Mason Dixon) released results for the last two weeks, only one of those (Research 2000) was conducted in the final week and even finished interviewing last Wednesday, a full six days before the election.
All the difficulties of sampling and modeling primary turnout are also present in Arkansas. The turnout for yesterday's Democratic primary represented just 20% of registered voters in Arkansas. It's possible that undecided voters "broke" to Halter and Morrison. It is also possible that sampling or the definition of likely voters led to an understatement of support for Halter and Morrison .
Beyond that, one aspect of the Arkansas contest is unique to primaries: Also-ran D.C. Morrison garnered 13% of the votes cast, and some of that vote may have come from voters wishing to express a preference for "none-of-the-above." If so, that is the sort of phenomenon that is more likely in a primary election where less rooted in the party affiliation of the candidates.
The bottom line is that primary election polling is much more difficult -- and more prone to error -- than polling in general elections. Either way, we are probably better off avoiding "rules" for poll interpretation and looking more closely at the evidence available for each contest.