I generally write about prediction markets or, aggregation of polls and fundamental data, but since we have been running some experimental mobile-based polling and it is extremely timely, I wanted to share the results.
The Pollfish survey, a weekly survey of 1,000 American voters, shows a very different trend from the general election polling of traditional surveys. Pollfish showed no Republican Convention bounce for Donald Trump and a solid Democratic Convention bounce for Hillary Clinton. In contrast, traditional polling, aggregated on Huffington Post’s Pollster, showed a bump for Trump of historically average magnitude coming out of his convention. We are still waiting for the polling from the Democratic Convention, which will come out in the next few days.
There are three key things that separate our survey from traditional election polls:
First, Pollfish, which is a mobile-only survey tool, uses a different mode than most polling. It is possible that people answer differently on mobile, because they are outside or because of the smaller screen. This is not to say one mode is right or wrong, but potentially different.
Second, the Pollfish survey makes no attempt to adjust its sample of respondents to their probability of responding, the basis of traditional polling. Instead, we correct biases via post-sampling analytics, by modeling and post-stratifying the raw data to the expected voting population.
Third, the Pollfish survey forecasts the sentiment of voters alone, instead of simultaneously estimating turnout and sentiment; thus changes in the Pollfish survey are more likely to reflect changes in sentiment more than changes in voter composition or changes in who showed up to the poll that day. Most polling first determines who the likely voters are and, in a second step, who they vote for. But, we model sentiment of likely voters in one single model, off of all Pollfish polls from 27 waves this year, with most of the emphasis on the last poll. Access to Big Data on all registered voters in the US helps us to focus on a stable space of likely voters.
A fourth difference? Our method is much cheaper and faster, allowing a team of researchers on a limited budget to track respondents weekly on a variety of issues. We are confident that modeling and post-stratification, combined with the large number of responses, allow us to successfully gauge public opinion. And, it is saying that Clinton is in solid shape heading out of both conventions.
This work is conducted jointly with my graduate students Tobias Konitzer and Sam Corbett-Davies, PhD candidates at Stanford University in communications and computer science, respectively.