06/24/2010 11:52 am ET Updated May 25, 2011

No evidence for Mellman's "magic" 50% approval

Last week, I criticized Matt Bai's claim that it was an "ominous sign" for Democrats that President Obama's approval rating is under 50%.

Writing in The Hill, top Democratic pollster Mark Mellman goes even further, calling 50% approval a "magic number" for midterm elections:

[P]erhaps there are some magic numbers after all. 

Take the effect of presidential approval on midterms. We graph the relationship between a president's approval rating and his party's gains and losses in midterm elections, thinking of the result as a smooth relationship. The lower the president's approval rating, the more seats his party loses.

But the pattern is not really so linear after all. There is a sharp discontinuity at 50 percent. Presidents whose approval rating is at 50 percent or above have lost, on average, just 11 seats in the House, while presidents under the 50 percent mark have lost an average of 33 seats.

Averages can obscure as much as they reveal, so pick apart the numbers. No president with an approval rating under 50 percent has lost fewer than 15 seats. The next smallest number is 26. Even a president just below 50 percent can lose a lot. When Democrats were punished with the loss of 52 House seats in 1994, President Clinton's approval rating rested just under the 50 percent threshold, at 48...

So where does President Obama stand? Last week Gallup put his approval at 45 percent, this week at 49. The weighted average of all polls says 46 percent. In short, for now, the president is hovering just below what may prove to be a magic number for Democrats in 2010.

Unlike Bai, Mellman brings historical evidence to the table, so let's consider his argument. He acknowledges the seemingly linear relationship between presidential approval and changes in House seats for the president's party in midterm elections. However, he argues that there is a "sharp discontinuity at 50 percent" in which "[p]residents whose approval rating is at 50 percent or above have lost, on average, just 11 seats in the House, while presidents under the 50 percent mark have lost an average of 33 seats."

First, let's replicate Mellman's numbers. (He appears to be excluding the replacement presidents -- Truman in 1946 and Ford in 1974 -- so I do the same here.) Using a cutpoint of 50 percent, I find that presidents with greater than 50 percent approval in the most recent Gallup poll before Election Day lose an average of ten seats and those below 50 percent lose an average of 33.5 seats (these slight discrepancies are likely the result of how different sources calculate seat change).

The problem is that the choice of 50 percent is arbitrary. For instance, presidents with approval ratings above 45 percent lost an average of 15 seats, while those below 45 percent lost an average of 33 seats -- results that aren't that different from Mellman's original numbers. Going the other direction, any cutpoint from 51 percent to 56 percent will yield the same results as 50 percent because there are no presidents who had approval ratings in that range in the data.

If you prefer graphical evidence, here is the data with a standard linear fit:


If we instead use a more flexible local polynomial fit to allow for nonlinearity, the predicted values show an inflection point around 50 percent approval, but the 95% confidence intervals reveal a great deal of uncertainty in that estimate -- hardly enough to justify a claim of a "sharp discontinuity":


With so few data points, it's very difficult to demonstrate a non-linear relationship. Absent further evidence, we can't be confident that a discontinuity exists at 50 percent.

It's also hard to believe the claim of a discontinuity in the context of the upcoming midterm elections. Given the state of the economy and the generic ballot, it's clear that Democrats are likely to lose a substantial number of seats regardless of whether Obama's approval rating is 49 percent or 51 percent on Election Day. Does Mellman believe otherwise?

Update 6/24 4:16 PM: To illustrate the point a bit further, I created a simple simulation of a linear relationship between approval and seat change that produces data approximately similar to what we observe above:


Over 1000 iterations of the simulation, the average outcome for presidents below 50 percent approval was a 12 seat loss while the average outcome above 50 percent approval was a 29 seat loss. In other words, it is very easy for a linear relationship to produce the sort of outcomes that Mellman describes.

[Cross-posted to]