As the third reference on Pollster in less than 24 hours, this post will amount to piling on, but I think it needs to be said: John Sides is right. The headline and lead on yesterday's Quinnipiac University poll does amount to "malpractice."
Here's the background. Yesterday, Quinnipiac released results from a new national survey of registered voters with the following headline:
April 21, 2010 - Obama's Bounce Goes Flat, Quinnipiac University National Poll Finds; But Voters Confident He Will Pick Good Judge.
Their lead paragraph explained:
President Barack Obama's job approval, which bounced slightly to a 45 - 46 percent split March 25 in the wake of his health care victory, has flattened out at 44 - 46 percent, his lowest approval rating since his inauguration, according to a Quinnipiac University national poll released today.
Like Sides, I have no grudge against Quinnipiac. They have always been exceptionally responsive and willing to disclose additional results and methodological details on request. Just yesterday polling director Doug Schwartz kindly responded to my email while on vacation to direct an assistant to respond to my request for party ID numbers in Florida. But the "Bounce Goes Flat" lead is just plain wrong. Here are Obama's job approval numbers as measured by Quinnipiac since January:
I'll let Sides explain:
There is no "bounce." There is no "flattening out." There is nothing but a big fat flat line. It doesn't make for a good press release, but that's the truth.
I'll be the first to concede that when writing about poll numbers it's easy to slip and describe small, non-significant differences as meaningful. I've certainly fallen into that trap myself from time to time. How many times can you write "not quite statistically significant" before readers' eyes start to glaze over? You can use "nominal" to modify "difference," but I'm guessing that few understand the implied technical meaning.
The headline and lead paragraph of a public polling release are something altogether different, especially when written by pollsters. After all, it's our job to get this stuff right. If we can't, how can we complain about the journalists and pundits who take our conclusions and run with them. Cases in point from yesterday:
Time's Mark Halperin: "Obama's Approval Flattens Out."
The Wall Street Journal's Washington Wire: "Just after signing the health bill, Obama's job approval rating ticked up, with 45% of voters approving and 46% disapproving. But the latest poll shows Obama's approval rating hasn't budged and now stands at the lowest level since he took office: 44% approve, 46% disapprove."
Daniel Foster on National Review's The Corner: "From Bounce to Dribble"
National Review's Jim Geraghty: "Quinnipiac: Health-Care Passage Moves Obama's Approval From 45 . . . to 44. Hey, look, it's the health-care-passage bounce! Nah, just kidding."
Those references leave out the scores of bloggers that simply quoted the entire lead sentence or paragraph.
If pollsters can get this sort of thing right, who will?