The Innovator's Take on Presidential Polls

As the U.S. presidential primary season approaches critical contests in Ohio and Texas tomorrow, pollsters are again taking a prominent place in the daily news. The unreliability of these polls provides an important lesson about innovation.

An unusually tight primary campaign has placed greater importance on frequently updated polling numbers. Different polls report different numbers, and polls change significantly from one day to the next. While this ensures pollsters get airtime on CNN, it makes it difficult for politicians and their advisors to make decisions based on poll results. Even polls administered 24 hours before an election can miss the mark.

Think about that for a second. How could scientifically derived estimates by professional pollsters modeling behavior that will take place 24 hours in advance be wrong?

There are three possible explanations for variations between poll results and voting results (beyond basic statistical error bands). One is a meaningful mismatch between the poll's sample and the population that actually votes. If the voting population deviates in meaningful ways from the sample population, results can deviate in meaningful ways as well.

The second explanation is that people don't accurately report what they are going to do. They might think it more "acceptable" to tell a pollster they are going to do one thing. But their actual behavior deviates from their stated behavior once it comes time to cast the vote.

Finally, people actually may change their minds between responding to a pollster and voting.

If politicians only trust the polls, they might end up making the wrong decision. What does this have to do with innovation? The corporate equivalent of a poll is market research. Almost every company that plans on launching a new product or service will run research to assess an idea's potential.

The problem comes when managers believe that the results of market research represent reality. Reality and research can deviate in powerful ways, particularly for disruptive innovations that offer fundamentally new bundles of benefits or create entirely new categories.

Maybe the survey population and the population interested in the disruptive idea bear little in common. Maybe consumers just can't visualize the idea. Dorothy Leonard has described eloquently how consumers struggle to respond to propositions on paper for things that don't yet exist.

Companies would be well served to remember the old saying, "Markets that don't exist can't be measured and analyzed."

That doesn't mean that companies should just fling ideas to the walls and see what sticks. They can do deep research on an opportunity space, seeking to understand whether they are addressing an important problem that can't be adequately solved today. They can let past patterns, intuition, and judgment guide their early activities. And they can push to get a version of their ideas into even a limited test market, to see what happens when real consumers have to pay real money for an idea.

Remember, consumers cast their real vote when they have to pay actual money for a product or service. If you are relying solely on a market forecast fed into a spreadsheet, be careful. Intuit Founder Scott Cook put it well when he said, "For every one of our failures, we had spreadsheets that looked awesome."

This post first appeared at Harvard Business Online.

Follow Scott Anthony on Twitter: www.twitter.com/sda222

Comments

[Cathexis]

Good point. Didn't Henry Ford once opine that "If I did what the peopel wanted, I'd be working on developing a faster horse!"?

[unitron]

"The unreliability of these polls provides an important lesson about innovation....There are three possible explanations for variations between poll results and voting results..."

I notice that you left out the possibility of the polls being accurate and the actual votes or abilities to vote having been tampered with in various ways.