An absurd amount of attention has been given today to a new Zogby internet poll. In a very well managed PR campaign, Zogby leaked the results last night before releasing them a bit after noon today, while also debuting a Zogby presence on Twitter. A little Googling will turn up many hits citing the not-yet-released-at-the-time results. Nate Silver at fivethirtyeight.com helped the poll get attention by attacking it before it was even released. The right is delighted-- "Obama approval drops below 50%!!" The left (with Silver's help) denounces the poll. Twitter is abuzz on both sides. And media, both old and new, couldn't contain their chatter over the results. (My favorite, the Boston Herald's Joe Dwinell's lede "The honeymoon is over, a national poll will signal today as President Obama's job approval stumbles to about 50 percent over the lack of improvement with the crippled economy." The fourth paragraph notes that "Some polls show Obama coasting with a 65 percent job approval, but not in Zogby's tally.")
It takes five seconds to put the new poll in perspective. Take that long to look at the chart above. The Zogby poll stands out pretty clearly in the chart, no?
To make a lot of the Zogby poll is to deliberately ignore the context in which it appears. Yes, the left is correct that it is an outlier. A huge one. The right looks desperate or ignorant by embracing this result as meaningful.
But rather than suppress the poll or ban it from polling analysis, let's just put it in the chart and let the data speak! No stats needed to get the point.
Poll results are far more meaningful when we look at them with a bit of perspective. The chart also makes clear how the Gallup and Rasmussen daily tracking polls differ from one another as well. Gallup's daily tracker consistently results in higher approval ratings than do non-daily national polls. Rasmussen consistently results in lower approval ratings than do non-daily national polls. But put them all together and we get a trend estimate based on all the polls that matches the trend for conventional non-daily polls quite well. If we pick only Gallup or only Rasmussen, we bias our understanding of the state of public opinion. Not by a huge amount, but by a persistent two-or-three points up or down compared to the overall trend or to the non-daily polls. This is no surprise. House effects, the tendency of polling organizations to produce modest differences from one another, are well known and much written about. By putting all the data in the chart for everyone to see we let these house effects stand out in an obvious way.
If someone still wants to cherry pick a result to suit their partisan preferences, then fine. That's called politics, duh! I'm not against it. But let's not confuse willful distortion for partisan purposes with analysis of what the data actually show.
The Zogby internet polls have a considerable history that I won't repeat here. The point is not to condemn them but to put them in perspective. That done, I don't think much more comment in required.