04/23/2014 05:31 pm ET Updated Dec 06, 2017

HUFFPOLLSTER: Senate Surveys Kick Off Another Round Of Poll Trutherism

Bill Clark via Getty Images

Four new polls of Southern states reignite the unskewed polling wars. We take another look at the trends shaping partisanship by generation. And "To be, or not to be?" is now a poll question. This is HuffPollster for Wednesday, April 23, 2014.

THE RETURN OF POLL 'TRUTHERISM' - Remember when conservatives dismissed 2012 polls showing Barack Obama leading because of a supposed "skew" toward Democratic voters in their samples, an argument that was ultimately discredited by Obama's victory? That trend is back, thanks to four polls released on Wednesday by the new polling partnership of the New York Times and the Kaiser Family Foundation. Let's start with HuffPollster's summary of the top line results: "Four southern Senate races with vulnerable incumbents -- Arkansas, Kentucky, North Carolina and Louisiana -- all remain closely contested, according to polls conducted by the Kaiser Family Foundation for The New York Times' The Upshot. Senate Minority Leader Mitch McConnell (R-Ky.) and Kentucky Secretary of State Alison Lundergan Grimes (D) are deadlocked at 44 and 43 percent, respectively, with Grimes leading by 6 points against Matt Bevin, McConnell's tea party challenger. In North Carolina, incumbent Sen. Kay Hagan (D-N.C.) is also roughly tied with two candidates, taking 42 percent to state Rep. Thom Tillis' 40 percent, and 41 percent to physician and conservative activist Greg Brannon's 39 percent….In Arkansas, the poll finds Sen. Mark Pryor (D-Ark.) 10 points ahead of Rep. Tom Cotton (R-Ark.)....In Louisiana, where candidates from all parties will run directly against each other, the poll finds Sen. Mary Landrieu (D-La.) at 42 percent, with her closest challenger, Bill Cassidy, at 18 percent. " [HuffPost, NYT]

Conservative pundit Bill Kristol's immediate reaction was to call the poll "bogus." Kristol took note of results to a question that asked respondents whether they voted in the 2012 presidential election, and if so, who they supported, writing: "The Times and Kaiser have produced a sample in Arkansas that reports they voted in 2012 for Romney over Obama--by one point. But Romney carried Arkansas in 2012 by 24 points. Similarly, the Kentucky sample is +3 Romney when reality was +23. The Louisiana sample is +3 Obama in a state Obama lost by 17, and the North Carolina sample is +7 Obama in a state he lost by 3. The whole point of question 12 is to provide a reality test for the sample. That's why they ask that question--we know what happened in 2012, so the only thing to be learned by asking the 2012 question of the sample is to ensure that it's a reasonably accurate snapshot of voters in the state. Of course there'll always be some variance between reality and the sample's report of its vote a year and a half ago--but not a 23 point variance. A reputable news organization would have looked at question 12 and thrown the poll out. But then again, it was the New York Times." [Weekly Standard, NYT questionnaire]

Republican pollsters and operatives echoed Kristol's complaint:

-Neil Newhouse: "Whoa. Those NYT polls might taste good with salt. Check out their '12 ballots - show O winning by 1.5; but he lost these 4 states by 13!" [@KCkid]

-Glen Bolger: "hey @chucktodd don't tell me u guys take the NYT polls in south seriously. Look at the percent of sample that voted Romney/Obama/didn't vote" [@posglen]

Karl Rove: " This is either incompetence or a deliberate attempt to boost the fortunes of Democratic candidates in trouble." [Fox News]

National Republican Senatorial Committee: The NYT/Kaiser polls have "deep methodological and reporting flaws that draw into question all of the findings, none more so than the Arkansas poll...the data is both unreliable and misleading." [NRSC]

-Logan Dobson: "A "Who did you vote for" question needn't match the results exactly. But being this far off? [link to image below]" [@LoganDobson]


Wrong comparison - The first problem with this latest bit of partisan poll bashing is that the numbers being compared aren't comparable. The NYT/Kaiser poll reported trial heat numbers among registered voters but results for 2012 voting among the larger population of all adults. "So it is no surprise," the Times' Nate Cohn writes in a response to the critics, "that so many respondents said they didn’t vote in 2012." In fact, the sampled adults tended to overreport their past turnout. The actual 2012 turnout among voting age adults in the four states was between 13 to 17 percentage points lower than what the respondents reported. This finding is also unsurprising. As HuffPollster wrote 10 years ago, "political scientists have consistently observed this pattern in surveys dating back to the 1940s and confirmed it with validation studies that check public records to see if individual respondents actually voted." [NYT, MysteryPollster]


Comparing apples to apples gets the results closer, but... Cohn recalculated the 2012 vote question filtering out those who said they are not registered voters and dropping those who said they did not vote in 2012. Based on those tabulations, "[t]he poll accurately captures Mr. Obama’s support, but tends to underestimate Mr. Romney’s performance. Mr. Obama’s share of respondents who say they voted in 2012 is startlingly close to his actual result. He won 48 percent in North Carolina, and 48 percent of self-reported voters say they voted for him. Mr. Obama won 38 percent of voters in Kentucky, and 38 percent of self-reported voters say they voted for him. Forty-two percent of self-reported voters say they voted for Mr. Obama in Louisiana; and 41 percent actually did. In Arkansas, 38 percent reported voting for Obama, and 37 percent did." (Had Cohn's calculations also filtered out the "don't know" category from the 2012 question, Obama's percentage of the vote would have been 2 to 3 percentage points lower than the actual result in Arkansas, Kentucky and Louisiana, but still nearly identical in North Carolina). [NYT]

Retrospective underreporting of the loser's support is typical - "Retrospective vote question is always difficult," says HuffPost's soon-to-be Senior Data Scientist Natalie Jackson via email. "People don't want to say they voted for a non-winner. My guess is that many Romney voters said they didn't vote rather than say the losing candidate. They may not have liked Romney but voted for him because of their party identification. It's easy to imagine them saying they 'didn't vote' 18 months later." It's even easier to imagine a lot of Romney voters refusing to share the name of the candidate they supported or saying that they voted for "someone else" other than Obama or Romney. In the NYT/Kaiser polls, between 7 and 12 percent recalled voting for "someone else," compared to just 1 to 3 percent in the actual results.

But wait, wasn't the Arkansas result something of an outlier? Yes. Most of the recent polls have shown a closer margin. Six of seven public polls conducted since late July have given Pryor a nominal advantage, although three were conducted by left-leaning groups (the seventh, from a Republican firm, showed a tie). HuffPost Pollster's model gives Pryor a lead of about 3 points. [Pollster chart]

Despite defending the results, the NYT's Cohn has some cautions for Democrats: "Mr. Pryor has a 10-point lead, according to the poll, but 16 percent of Mr. Pryor’s supporters — or 8 nearly percent of all voters — oppose the Affordable Care Act and say they could not vote for a candidate who disagrees with their stance on the issue. Mr. Pryor, of course, voted for the Affordable Care Act. If those voters flip, his opponent, Representative Tom Cotton, will have the advantage. Other Democrats face a similar challenge: In every contest, at least 10 percent of Democratic supporters oppose the Affordable Care Act and say they wouldn’t vote for a candidate who disagrees with their stance. All four Democratic Senate candidates in these states support the law….The poll hints at the looming turnout challenge for Democrats. Voters who said they would definitely vote this November were more favorable to Republicans than registered voters as a whole." [NYT]

A few other takes on the NYT/Kaiser survey:

-Dylan Scott notes favorable numbers for politicians backing the ACA. [TPM]

-Jonathan Bernstein says the poll has value as an outlier. [Bloomberg]

-Sean Trende thinks nitpicking polls internals is "dangerous business." [@SeanTrende here and here]

-Sasha Issenberg restates his doubts about self-reports of past and future voting. [@victorylab here and here]

-HuffPost Poet/Pollster Emily Swanson takes on the latest results, in verse [@PollSonnets]

HUFFPOLLSTER VIA EMAIL! - You can receive this daily update every weekday via email! Just click here, enter your email address, and and click "sign up." That's all there is to it (and you can unsubscribe anytime).

IS PARTISANSHIP SHAPED BY PRESIDENTIAL PERFORMANCE IN VOTERS' EARLY YEARS? Following up on a Tuesday post] by Dan Hopkins about the early onset of partisan loyalty, political scientist Alan Abramowitz writes: "Yes, partisan orientations of younger voters will be shaped to some extent by performance of the president during their adolescent and early adult years. But aren't you ignoring another key factor that is operating today--one that wasn't present for earlier generations entering the electorate, or not nearly as much--demographic change. A big part of the reason why younger voters are much more Democratic than older voters today is not different experiences when they entered the electorate but differences in race and ethnicity. The nonwhite share of voters under age 30 is much greater than the nonwhite share of voters over age 60."

Abramowitz also charted the relationship between party ID and age, controlling for race:

Hopkins responds - "My piece doesn't dispute (and indeed fully acknowledges) the changing demographic composition of the electorate, noting: 'Likewise, it’s tempting to credit broader demographic changes, as younger voters are a more ethnically and racially diverse group. But that can’t be the whole story; the same trends are evident among non-Hispanic whites.'...It also is careful to differentiate non-Hispanic whites from others: 'The figure below shows the results for just over 1,800 non-Hispanic white voters, although those for non-white voters are surprisingly similar.' In short, my piece is about a different phenomenon, and doesn't at all contradict the clear demographic evidence that younger generations are far more ethnically and racially diverse. It simply says that's not the whole story."

POLLING SHAKESPEARE - HuffPollster, with Emily Swanson: "William Shakespeare would have turned 450 on Wednesday. In honor of his birthday,a HuffPost/YouGov poll posed some of his most memorable questions to the American public….Hamlet's famous query 'To be, or not to be?' elicited a fairly clear answer from Americans: 50 percent prefer to be. Just 5 percent opted not to be, while 45 percent weren't sure, but admitted 'that is the question' (we didn't give them any other option)....Republicans (43 percent) were more likely than Democrats (37 percent) or independents (37 percent) to say they had achieved their own greatness. On the other hand, Democrats (17 percent) were more likely than independents or Republicans (11 percent for both) to say they'd been born great. " [HuffPost]

WEDNESDAY'S 'OUTLIERS' - Links to the best of news at the intersection of polling, politics and political data:

-Quinnipiac and PPP (D) find Hickenlooper leading in the Colorado governor's race. [Quinnipiac, PPP]

-Two polls find a tight race for Senate in Colorado. [Politico]

-PPP (D) finds a close race for Wisconsin governor. [PPP]

-Retirement remains Americans' top financial worry. [Gallup]

-Taxpayers are troubled by inequality, Mark Mellman says. [The Hill]

-David Hill sees Democrats as using medical marijuana and minimum wage as issues to drive turnout. [The Hill]

-Kim Yi Dionne and Boniface Dulani share a primer on Malawi's upcoming elections. [WashPost]

-Do April showers really bring May flowers? [538]

-A statistician reviews the latest findings on his most important data point -- his 6-month old son. [Flowing Data]



Senate Majority Leaders Through The Years