Nate Silver begins his rebuttal to Robert Moran this morning by saying, "I don't like criticizing our good friends over at Pollster.com." Well, I don't like criticizing our contributors either, but when Bob wrote on Tuesday in the context of a post on the latest results on the Ohio governor's race that incumbent candidates "get what they get in the tracking, " that it's a "fairly ironclad rule" that "incumbents tend to get trace elements of the undecideds at the end of a campaign," Nate is right and Bob is wrong.
I sympathize with Bob because my instincts led me to a similar line of argument when I started blogging five years ago. My experience as a campaign pollster, gained mostly during the late 1980s and 1990s, taught me to expect the sort of "incumbent rule" patttern that Bob refers to. It was not just an impression. Nick Panagakis, the long time pollster for the Chicago Tribune and other Midwestern media outlets, published some evidence on the Rule in a 1989 article in The Polling Report. Progressive blogger Chris Bowers updated the Panagakis data in 2004, and I summarized both a few months later. When I saw five polls in Ohio showing amazing consistency in Bush's number (46% to 47%), with considerable variation in the Kerry number (45% to 50%), I argued that we were seeing the "underlying principles of the Incumbent Rule in action."
Problem was, I was wrong. Both Bush and Kerry got a bigger percentage of the vote on Election day than they had received in the polling averages (Bush gained a little bit more in Ohio, but not much). Moreover as Bowers had, to his credit, already flagged in 2004, the Rule had been "weakening" since 1998, and by 2006 it was clear that it had largely vanished in competitive, statewide races (the same post includes comments from four campaign pollsters on why they think the "doctrine" was no longer valid). Contests like the 2009 New Jersey Governor's race have been the rare exception, and not a rule.
Since 2004, however, I have also realized that when pollsters or political junkies cite the Incumbent Rule, they sometimes mean two different things. The topic I obsessed over in 2004 involved whether we should anticipate a "break" among undecided voters toward the challenger between the final round of polling and Election Day. Bob's argument on Tuesday, however, argues something a little different: That the incumbent's percentage is unlikely to rise during the course of the campaign, that you get on Election Day what you're getting in tracking, even as far out as February.
Nate's post -- which is well worth reading in full -- attacks the second idea, and I want to stay focused on that topic for the rest of this post. He looks at 63 elections for Senate and Governor since 2006 in which there were polls conducted between January and June and where the two major party candidates ultimately won at least 90% of the combined vote. He then calculated a simple average of all polls fielded between January and June and compared those to the election results.
Nate finds that it is "extremely common for an incumbent to come back and win reelection." He finds that 19 of 30 incumbents who scored under 50 percent in the average of early polls ultimately won reelection. Further he finds that in almost every case (58 of 63) the incumbent ended up with a larger percentage of the vote than they had received in the early polling average. Those findings contradict Bob Moran's argument. .
However, Nate's data also suggests a middle ground and confirms that campaign pollsters and other political professionals are right to focus more on the incumbent's percentage than that of the challenger in early polling. First, as Nate puts it, the "corollary of Bob's hypothesis is almost always true." The incumbent won in 32 of 33 cases since 2006 where early polling showed the incumbent with more than 50% of the vote. Further, when the incumbent's percentage fell under 45%, their probability of succeeding dropped dramatically: Only 5 of 15 ultimately won. Finally,
[I]t does appear to be the case that the incumbent's share of the vote is a better predictor of the final voting margin than the challenger's share. The correlation between the incumbent's vote share in early polls and the final voting margin is .85; the correlation between the challenger's vote share and the final margin has a smaller magnitude, at (negative) .80. Interestingly, the correlation between the margin in early polls and the final margin is also just .85 -- no better than that obtained from looking at the incumbent's vote share alone. This may suggest that the opponent's vote share provides little additional informational value once the incumbent's vote share is known. As I hope I've made clear, however, this does not mean that incumbents "get what they get in the tracking"; they almost always add to their number.