We examine the widely varying polling methods being used in Iowa. The polling misfire in Greece raises questions about the accuracy of polling yet again. And is Trump really the frontrunner? This is HuffPollster for Friday, July 10, 2015.
KNOW YOUR IOWA POLLSTERS - With retail campaigning just getting underway in Iowa, the preferences of the state's likely caucus goers will soon start to matter. Caucus turnout is low compared to typical primary elections, so the details of how pollsters select those voters are important. To help give polling junkies a sense of the differences in their methods, we introduce "Know Your Iowa Pollsters," a running HuffPollster feature to provide some of the most basic details.
Today we examine the methods used by the five most recent Iowa polls, all conducted between late May and late June, that collectively span a wide spectrum of approaches. No two use exactly the same combination of mode and sampling technique. Two use live interviewers exclusively (Des Moines Register/Selzer and Quinnipiac University), two use an automated recorded voice method (Monmouth College/KBUR/We Ask America and Advantage Inc) and one uses a combination of live telephone and an internet panel sample (Morning Consult).
The DMR/Selzer and Quinnipiac surveys sampled both landline and mobile phones. The two automated surveys are prevented by the federal TCPA from dialing mobile phones. The Morning Consult poll sampled landline phones for its telephone component and attempted to reach cell-only households through the internet panel.
Mobile phone usage is a big issue in Iowa, perhaps even more than other states. The latest state level data from CDC/NHIS estimated that 48 percent of Iowa households had only wireless telephone service in 2013, while another 17 percent said they answer most calls on a wireless phone despite having a landline at home. And that estimate is already 18 months old. The national estimates for 2013 were lower (40 percent cell only, 17 percent cell mostly).
Despite the significant variation in methods, the five most recent polls of Republicans found very little variation in support for Scott Walker, who led with 17 to 19 percent of the vote. Support for Jeb Bush was also relatively consistent at between 8 and 12 percent. Donald Trump's support was higher on the two polls conducted after his June 16 announcement (10 and 7 percent) than those that came before (2 to 5 percent).
The most recent polls of Democrats showed more variation, but with the timing of Bernie Sanders' entry into the race likely obscuring any differences owing to methodological variation.
Des Moines Register/Selzer
- Most recent surveys: Republicans, May 25-29; Democrats June 19-22.
- Survey mode: Telephone, live interviewer.
- Sample: Registered voters with landline and mobile phones selected from Iowa's secretary of state voter registration list.
- Likely caucus-goer selection - Ask and confirm that respondents are registered to vote in Iowa, then select those who say they will definitely or probably attend the caucuses using the following question. "How likely is it you will attend one of the caucuses scheduled for February of 2016—will you definitely attend, probably attend, or probably not attend? (If definitely or probably attend, ask:) Will you attend the Democratic or the Republican caucus?"
Quinnipiac University
- Most recent surveys: Republicans (methodology) and Democrats (methodology), June 20-29
- Survey mode: Telephone, live interviewer.
- Sample: Dual samples of landline and mobile telephone numbers generated using Random Digit Dialing.
- Likely caucus-goer selection - Ask and confirm that respondents are registered to vote, then select those who say they will definitely or probably attend the caucuses using the following question: "How likely is it you will attend one of the presidential caucuses scheduled for February of 2016 -- will you definitely attend, probably attend, or probably not attend? (If definitely or probably attend, ask:) Will you attend the Democratic caucus or the Republican caucus?"
Monmouth College/KBUR/We Ask America
- Most recent surveys: Republicans and Democrats, June 27-29.
- Survey mode: Automated, recorded voice telephone (IVR)
- Sample: Registered voters with landline phones only selected from Iowa's secretary of state voter registration list.
- Likely caucus-goer selection - Based on responses to the following question: "Do you plan to participate in the Democratic Caucus, the Republican Caucus or no caucus at all?"
Morning Consult
- Most recent surveys: Republicans and Democrats, May 31-June 8.
- Survey mode: Combination of online (n=644) and live interviewer telephone (n=261).
- Sample: Landline telephone numbers were sampled using Random Digit Dialing. Online interviews were conducted by Research Now and drawn from an "opt-in" research panel.
- Likely caucus-goer selection: Survey did not screen for likely-caucus goers. It reported for all self-identified partisans of the respective parties (i.e. questions about the Democratic candidates were asked of all voters who said they are registered as Democrats, questions about the Republican candidates were asked of all who said they are registered as Republicans).
Advantage Inc.
- Most recent surveys: Republicans only, June 8-10.
- Survey mode: Automated, recorded voice telephone (IVR)
- Sample: Registered voters with landline phones only selected from Iowa's secretary of state voter registration list.
- Likely caucus-goer selection - Selected self-identified registered Republicans who said they were likely to participate using the following question: "What is the likelihood of your participation in the upcoming Republican Presidential Caucus -- are you extremely likely, very likely, somewhat likely, or not very likely at all?"
POLLS AS FIRST PRIMARY? - Jennifer Agiesta: "Four weeks from Thursday night, a group of at least 10 Republican candidates for president will take the debate stage in Cleveland, Ohio, with polling playing a major role in determining which candidates out of the ever-expanding field get to be behind a podium -- and which will be watching from home. Fox News, which is co-sponsoring the debate with Facebook, has said that **those on the stage must 'place in the top 10 of an average of the five most recent national polls, as recognized by FOX News** leading up to August 4th at 5 PM/ET. Such polling must be conducted by major, nationally recognized organizations that use standard methodological techniques.'...Until Fox releases a more detailed description of which polls meet its standards, it's impossible to know which group of 10 will stand before the cameras on August 6. But if it were held today, it's clear that those candidates averaging 2 points or less would be left out." [CNN]
Yet imprecise at measuring single-digit support - Steven Yaccino: "[B]ecause of the varying sample sizes, margins of error, and targeted respondents featured in different national polls, the winners and losers of this new debate primary season may have little relation to their prospects of becoming the eventual nominee. Methodologically, they might as well be drawing straws.'A microscope has not yet been invented that enables us to determine the difference between the 10th-place person and the 11th-place person,' said Ken Goldstein, a professor of politics at the University of San Francisco and a polling analyst for Bloomberg Politics. [Bloomberg]
Why such imprecision? - Darrel Rowland: "Joe Lenski, a former officer of the American Association for Public Opinion Research, listed several problems with combining poll results: Not every serious potential candidate is included in every poll....The percentage of undecided voters in current polls ranges from around 3 to 16 percent, indicating that some survey administrators are pushing respondents to make a decision, but others are not...The polls have different numbers of respondents; and some include 'likely' voters, others just 'registered' voters....National surveys contain varying margins of sampling error....Some measures arbitrarily eliminate business magnate and television personality Donald Trump; others include him. Lenski foresees lawsuits, possibly involving polling firms being asked to justify their methodologies, from candidates who don’t make the final 10." [Columbus Dispatch]
-Political insiders don't want to use polls to determine debate participants. [National Journal]
TRUMP AS FRONT RUNNER? NOT SO FAST - Natalie Jackson: "[I]n in the wake of comments he made about Mexican immigrants...[businessman Donald Trump's] polling numbers have increased exponentially, putting him near the top of the crowded Republican field...Some pollsters and media outlets will claim a candidate is 'leading,' or in second place, or even a 'frontrunner.' However, under most circumstances, a candidate polling at 12-15 percent would be laughable -- and have little chance of winning. In that light, all of the Republicans are doing pretty abysmally in the polls...[I]t’s important to keep movement in the polls in perspective. It’s important to remember that early polls are not good at predicting outcomes. What the polls are providing is a metric of where opinion is right now -- and it's all over the place." [HuffPost]
INVISIBLE PRIMARY: FEW GOP ENDORSEMENTS SO FAR - John Sides: "[In 2012, C]ompared with previous years, relatively few Republican governors, U.S. senators, or members of the U.S. House endorsed a candidate before the Iowa caucus....We’re now six months into 2015, and the same pattern is evident. The graph above shows that about 90 percent of key Republican leaders have yet to endorse a candidate. The percent who have endorsed is only slightly higher than at this point in 2011. Of the 33 endorsements that we have counted, Jeb Bush has more than any other candidate (14), mostly because of endorsements from some Republican members of Congress from Florida." [WashPost]
-Among Democrats: Dozens of house liberals have endorsed Hillary Clinton; none have endorsed Bernie Sanders (so far). [The Hill]
WHY THE GREEK POLLS MISSED - HuffPollster: "Sunday was a bad day for pollsters, adding yet another poor performance following a string of recent high-profile polling misfires. This time the final round of polls on the Greek referendum showed the bailout deal failing by 3 to 4 percentage points, but the actual vote was not nearly as close. The 'no' side prevailed by a 22-point margin -- 61 to 39 percent. To be fair, polls on the referendum faced an unusually high degree of difficulty. Greek Prime Minister Alexis Tsipras called for the special vote just eight days before it occurred. In between, Greece failed to meet an International Monetary Fund deadline, effectively putting itself into default and creating a genuine crisis. The lengthy, technical ballot language added another element of complexity... Although definitive conclusions about why the polls missed in Greece will come only after the pollsters thoroughly investigate their methods and data, there are signs that the same challenges facing survey-takers elsewhere may have made a bad situation worse in Greece. Here are some reasons why the polls in Greece erred:.." [HuffPost]
WHY U.S. POLL FORECASTS COULD MISS IN 2016 - Sean Trende: "The sudden emergence of Nate Silver of FiveThirtyEight.com in 2008 helped to bring scores of previously obscure electoral forecasters to the forefront of American culture, and enabled dozens of writers with similar interests to make careers out of what looked like a hobby. But the amount of faith the public now puts in us is misplaced. Electoral modelers have a nerdy little secret: We aren’t oracles. Draw back the curtain, and you’ll see that we are only as good as the polls we rely on and the models we invent. And there are real problems with both. That’s why the 'data journalism' movement contains the seeds of its own destruction. The danger lies in data journalists’ tendency to belittle skeptics and other analysts who get it wrong. Worse is the distinct tendency to downplay how much uncertainty there is around our forecasts. This is a shame, because sooner or later—probably sooner—the models are going to miss in an American presidential election and data journalism as a whole is going to suffer." [Politico]
THIS WEEK'S NATIONAL POLLS
-YouGov finds Donald Trump "leading" the GOP field in a national poll. [YouGov]
-Democrats views on the confederate flag grow increasingly negative. [Gallup]
-Sixty percent of Americans now say they support the framework of Obama's nuclear deal with Iran. [The Chicago Council]
-Americans generally approve of the Supreme Court's decision to uphold the use of Oklahoma's controversial lethal injection drug in executions. [HuffPost]
-Hillary Clinton continues to remain the top choice candidate for most Democrats. [HuffPost]
HUFFPOLLSTER VIA EMAIL! - You can receive this weekly update every Friday morning via email! Just click here, enter your email address, and click "sign up." That's all there is to it (and you can unsubscribe anytime).
THIS WEEK'S 'OUTLIERS' - Links to the best of news at the intersection of polling, politics and political data:
-Nate Cohn sees Bernie Sanders hitting a wall of rank-and-file Democratic support for Hillary Clinton. [NYT]
-Nate Silver explains why Bernie Sanders could win Iowa and New Hampshire and lose everything else. [538]
-The 2016 terrain looks a lot like 2012, writes Dante Chinni. [WSJ]
-Sasha Issenberg chronicles Republican efforts to close the "campaign science gap" with the Democrats. [Bloomberg]
-Eitan Hersh offers the "real story" on how data-driven campaigns target voters. [WashPost]
-Ted Cruz's presidential campaign will work with a "psychographic" targeting firm. [Politico]
-Frank Newport considers that the way two recent Supreme Court decisions fit and don't fit with public opinion. [Gallup]
-Steve Vancore rates the latest Gravis poll of Florida as deserving "between half and full shaker of salt." [SaintPetersBlog]
-Pew Research explores what would happen if they stopped offering a $5 incentive to cell phone respondents. [Pew]
-SurveyMonkey names a new CEO. [NYT]
-A new polling venture aims to conduct "policymaking simulations" with sampled voters. [Baltimore Sun]
-Kaiser Fung praises a innovative effort to visualize survey results. [Junk Charts]