Xbox, in conjunction with the polling company YouGov, has launched a new poll for this election cycle. The goal of the poll is threefold. First, Xbox wants to create an engaging product for its users. Second, the polling team wants to be able to provide meaningful insight into the 2012 election. Third, both Xbox and YouGov recognize the research angle of this work; this poll is the product of cutting-edge interactive television and polling techniques that, while not supplanting the established methods today, will be instrumental in both entertainment and information gathering in the future.
In the interest of full disclosure, as a member of Microsoft Research in New York City, I am the head pollster on the Xbox side of this team.
There are two connected polling operations: the daily poll and the live event polling. First, every day on Xbox Live's election channel there is a new poll that asks between three and five questions. New users answer a slate of standard background questions the first time they enter. Every day the poll asks the standard voter intention question and then groups the remaining questions into daily themes: economy, social issues, etc. Second, Xbox is streaming the pool feed for all four debates and asks question on the bottom of the screen periodically to their users during the debate. A few baseline questions at the beginning are followed by a stream of questions reacting to the segments. In both polling settings respondents self-select into the polls.
Above is an example of the live event polling on the Xbox Live from Thursday's VP debate. Click on the image to watch the video!
Respondents are enjoying the experience; they are using it in droves and coming back. First, over 20,000 unique respondents per day are answering the daily polls and there were over 30,000 unique respondents answering each question during Thursday's VP debate. Second, Xbox is getting very high repeat engagement with upwards of 20 percent of users taking the daily poll any given day having already taken a daily poll in the past. And, the engagement levels during the debates have been at least a magnitude larger than standard engagement for web-based polling.
After Thursday night's VP debate Xbox was able to release timely and meaningful data on the reactions from their viewers. The sheer quantity of the respondents answering each question provides us with a very meaningful number of undecided and leaning respondents. Biden dominated Ryan in all of our top-line questions; undecided registered voters thought he won the debate, was more truthful and more capable of being president. Similarly, respondents leaning toward Obama were more overwhelming for Biden in all three questions than respondents leaning toward Romney were for Ryan. We have no prior reason to believe that the undecided voters in our poll behave too differently from average undecided voters, but since this a self-selected non-representative sample, further research is ongoing. Yet, parsing the data by both age and gender, we saw perfectly consistent results.
Above is a snippet of results from Thursday night's Xbox Live/YouGov poll. Approximately 30,000 Xbox Live users responded to each question posed during the debate on 10/11/12. The survey was designed and analyzed by YouGov.
Expect a regular, meaningful stream of results from the Xbox Live/YouGov poll between now and Election Day. Harnessing both the large sample sizes and repeated respondents, we are busy learning about our repeated users from our unbalanced panel that have switched over the last few weeks. We are also exploring the value of randomly generated subsamples of daily respondents. While, we embrace the oversampling we have hard to get demographics, like 18-29 males and young families.
While I am happy to help create a fun and meaningful product for 2012, what really excites me about this poll is the research and potential of this technology. I am huge believer in the value of the probability based scientific polling and I use the aggregated totals from HuffPost's Pollster as a key ingredient in my real-time forecast of the 2012 election. But, my job is to engage in academic, peer-reviewed style research that pushes the boundaries of information aggregation.
One of those boundaries is to see how far we can increase the value of information supplied in polls of self-selecting and non-representative respondents. We hope that by gathering massive amounts of respondents and having repeated responses by many, we will be able to overcome some of the statistical biases inherent in our sample. We are experimenting with new weighting, new debiasing methods, new questions, and new interactive interfaces. We recognize the limits and embrace the advantages: quickly updating questions, massive coverage, and cost-effective respondents.
So many people today watch TV with a second screen around and we are giving them something fun, related, and meaningful to do with extra energy; we have enacted the promise that is interactive television with the hope that it will lead us towards the future of information aggregation.
If you have an Xbox, watch live during Tuesday's debate. Either way, I look forward to updating you on the results.