Nate Cohn takes another whack at PPP. And, probably not for the last time, new polls show a close race in Virginia and a looming blowout in New York City. This is HuffPollster for Thursday, September 19, 2013.
NATE COHN'S PPP HITS KEEP COMING - Nate Cohn resumes his case against Democratic Party affiliated, automated pollster Public Policy Polling (PPP), adding some new findings: “Even when there is a polling average, PPP tends to fall farthest from a polling average when it’s toughest to weight a poll to match the average. What situations are those? When a pollster needs to produce two accurate results, not just one. That happens, for instance, in states with both a presidential contest and a down-ballot race. So there’s exactly the pattern you’d expect if they were weighting toward the polls: (1) PPP nails the presidential results but sacrifices the close senate races in the presidential battlegrounds, and (2) PPP nails the close senate and gubernatorial contests but badly misses the presidential margin in states with a clear favorite for President. So, for instance, PPP nailed the presidential race in Florida, but missed the senate race by 8 points; conversely, PPP nailed the gubernatorial contest in Washington, but missed the presidency by 7 points. In other words, it just looks like PPP can't have it both ways.”
[TNR]
Overkill? - Public reactions to Cohn's piece were mixed among pollsters and poll watchers. "Here we go again," tweeted Marist Institute Senior Analyst Natalie Jackson, who had been skeptical of PPP following Cohn's first piece. "I agree that results don't justify bad methodology," she added, "but this feels like beating a dead horse." Political scientist Drew Linzer expressed frustration that Cohn's attacks imply "that there's only one right way to poll, and everyone does it that way but PPP." [@NatalieMJ84, @DrewLinzer]
Or 'devastating?' - On the other hand, former DCCC data/targeting director Aaron Strauss, who last week defended PPP's methodology as "pretty solid," had this reaction: "[Cohn's] Pres v Sen/Gov table is a devastating blow to @ppppolls. Shows how subjective their weighting must be." [@Aaron_Strauss here and here]
But mostly, non-pollsters ask, what's the big deal? - "No," Cohn argues at the top of his latest article, "the 'goal' of a poll is not to 'get it right,' or another variant of the point that ‘PPP gets the results right, so I don’t see what the problem is.’" Yet for many, especially beyond the small community of pollsters and academic survey specialists, those sentiments are the core of the problem. Cohn's arguments are complex and sometimes arcane, and non-pollsters just want to know if poll results are accurate.
Maybe it's time to rethink traditional measures of accuracy. The problem is that we're basing assessments of accuracy on comparisons of a pollster's final poll to the election result. One thing implicit in Cohn's analysis is that such accuracy measures can be gamed: "PPP is basically saying 'yes, our methodology reserves us the power to do whatever we want [potentially including weight to polling averages], but you should trust us because our results are good.' Well, your results would be good, wouldn't they?" So for now, at least, consider this advice from a series of tweets by Democratic pollster Nick Gourevitch: "Whether or not you agree with @Nate_Cohn on PPP, the point that accuracy isn't everything is a good one...Especially when comparing polls taken a night or two before Election Day to polls weeks/months before. Campaigns happen & polls change...Polling is messy & expertise critical, but accuracy isnt everything & methods matter. [So] its possible to agree w/ both @DrewLinzer & @Nate_Cohn." [@NickGourevitch here, here and here]
POLL SHOWS CLOSE RACE IN VIRGINIA - Roanoke College, which put Cuccinelli 6 points ahead in July: “Democrat Terry McAuliffe holds a statistically insignificant lead over Republican Ken Cuccinelli (35%--33%), while 22 percent of likely voters in Virginia remain undecided in the 2013 Gubernatorial election, according to The Roanoke College Poll. Libertarian candidate Robert Sarvis claimed 8 percent of respondents.” [Roanoke]
DE BLASIO HAS HUGE LEAD AGAINST LHOTA IN NYC - Quinnipiac: “There are yawning racial gaps on many issues as Public Advocate Bill de Blasio, the Democratic candidate for New York City mayor, leads Republican candidate Joe Lhota on every measure except taxes and has a commanding 66 - 25 percent lead among likely voters, according to a Quinnipiac University poll completed last night and released today. Independence Party candidate Adolfo Carrion gets 2 percent, the independent Quinnipiac University poll finds. De Blasio leads 52 - 40 percent among white voters, 90 - 3 percent among black voters and 68 - 18 percent among Hispanic voters. He leads 63 - 29 percent among men and 69 - 21 percent among women and has leads of 30 percent or higher among every income, education and age group.” [Quinnipiac]
HUFFPOLLSTER VIA EMAIL! - You can receive this daily update every weekday via email! Just enter your email address in the box on the upper right corner of this page, and click "sign up." That's all there is to it (and you can unsubscribe anytime).
THURSDAY'S OUTLIERS' - Links to more news at the intersection of polling, politics and political data:
-Sabato's Crystal Ball predicts a small gain for House Republicans. [Center for Politics]
-A new WBUR/MassINC poll finds a top tier emerging from a field of 12 mayoral candidates in Boston. [WBUR]
-Mass. Numbers rounds up the five polls released to date on the Boston mayoral race. [Mass. Numbers]
-North Carolinians’ approval of President Obama and Gov. Pat McCrory drops. [Elon]
-Pew Research finds the defunding Obamacare strategy is "in step with" Republican primary voters. [Pew Research]
-Karl Rove argues the "defunding" strategy is self-defeating for the GOP. [WSJ]
-Chris Cilizza and Sean Sullivan find a 'Big Lebowski' analogy in debt ceiling poll results. [WaPost]