(Thanks to Vint Cerf, Jacob Ziv and Marc Smith for advice and discussions)
Who doesn't want the survey-less society? It may be the next big thing after the paperless office. Web 2.0 is making it happen. Just like with the paperless office, it will be another paradigm shift for society. It will change the way we make decisions together.
So much is driven by analytics in this era of computing, all marketers live by it. Their thirst for data just grows they more they get. They have become dataholics. Ordinary people are increasingly nagged to answer surveys. It's soon not possible to make an ordinary phone call without opting in or out of a survey before being let through to a human being. But the value of that data is questionable. And, hopefully, these surveys will be redundant soon.
There is accumulating so much data in the IT-clouds, that probably most trends are to be found in what already is recorded. The good thing with this data is that it covers everything, it isn't limited to a narrow window of interest. Everything that people talk about is there.
The strategists who until now are working with surveys are always looking for the answer to one question: "who says what?".
Surveys assume which are the important groups of people and then go check what they say, or the other way around - they assume which are the important ideas and then check who say them. The answers are won't be better than the questions. But asking the right questions is really tricky. You don't know what you don't know, it's a lot of guessing. That will not do in the innovation society, when every important trend is driven by the introduction of things that never existed before.
It's time to go beyond surveys and other analytics that can't spot new trends without first assuming their existence.
Can it be done? Yes, it can, using the data in the clouds and introducing new ways of thought.
The picture with the red-blue scale is an example of the next generation analytics. It finds trends, rates and ranks - it even quantifies bias - without making any assumptions about any groups of people, or what they think or what they do.
The example shows the most discussed blogs, from blue (Democrat) to red (Republican) discussing Obama's State of the Union address. The choice of media and the assignment of political color was done by a general algorithm, which does not care about politics or media.
We simply fed "state of the union" into the algorithm and this is what came out.
We did NOT assume the existence of Democrats or Republicans. We did NOT assume anything about blogs or news outlets, nor select any to look at, nor guess political sympathies. We did not assume which opinions or sentiments mattered.
How do we do it? The understanding of how to do it comes from a concept I have named 'commemities'. The algorithm was done by combining two modern, usually separate, analytics approaches - social network analysis and search/semantics. It is a natural thing to combine them because social network analysis shows how people connect and semantics shows how words connect, while the reality is that people and words connect each other.
1) Words are connected by people. People are connected by words. 2) A community has a shared language. A shared language has a community. 3) The community and the shared language can be seen as one: the "commemity" - the community and its memes.
Now we can explain how we did the list. We make a social network analysis of Twitter, finding two big clusters chatting about the "state of the union". These two clusters are Democrats (red) and Republicans (blue). Now we determine their shared language, and we will have mapped the commemities. One way of doing it is to check the web pages they refer to. There are blue referrals and there are red referrals. We sum up the referrals to the web pages and mix the colors. As expected, Fox is republican language. The democratic congressional campaign committee is democrat language.
This simple notion of commemities enables new analytics tools for strategic analysis, marketing, governance and planning, that can reveal unknown disruptive trends and answer unasked questions.
Controlling the socio-semantic data means power. Companies like Google, Facebook, Amazon, linkedIn, Twitter are obvious players. Cellular network operators are, too, they are in charge of the data flows between our handsets and the clouds, the nerve system of collective intelligence.
Our 2024 Coverage Needs You
It's Another Trump-Biden Showdown — And We Need Your Help
The Future Of Democracy Is At Stake
Our 2024 Coverage Needs You
Your Loyalty Means The World To Us
As Americans head to the polls in 2024, the very future of our country is at stake. At HuffPost, we believe that a free press is critical to creating well-informed voters. That's why our journalism is free for everyone, even though other newsrooms retreat behind expensive paywalls.
Our journalists will continue to cover the twists and turns during this historic presidential election. With your help, we'll bring you hard-hitting investigations, well-researched analysis and timely takes you can't find elsewhere. Reporting in this current political climate is a responsibility we do not take lightly, and we thank you for your support.
Contribute as little as $2 to keep our news free for all.
Can't afford to donate? Support HuffPost by creating a free account and log in while you read.
The 2024 election is heating up, and women's rights, health care, voting rights, and the very future of democracy are all at stake. Donald Trump will face Joe Biden in the most consequential vote of our time. And HuffPost will be there, covering every twist and turn. America's future hangs in the balance. Would you consider contributing to support our journalism and keep it free for all during this critical season?
HuffPost believes news should be accessible to everyone, regardless of their ability to pay for it. We rely on readers like you to help fund our work. Any contribution you can make — even as little as $2 — goes directly toward supporting the impactful journalism that we will continue to produce this year. Thank you for being part of our story.
Can't afford to donate? Support HuffPost by creating a free account and log in while you read.
It's official: Donald Trump will face Joe Biden this fall in the presidential election. As we face the most consequential presidential election of our time, HuffPost is committed to bringing you up-to-date, accurate news about the 2024 race. While other outlets have retreated behind paywalls, you can trust our news will stay free.
But we can't do it without your help. Reader funding is one of the key ways we support our newsroom. Would you consider making a donation to help fund our news during this critical time? Your contributions are vital to supporting a free press.
Contribute as little as $2 to keep our journalism free and accessible to all.
Can't afford to donate? Support HuffPost by creating a free account and log in while you read.
As Americans head to the polls in 2024, the very future of our country is at stake. At HuffPost, we believe that a free press is critical to creating well-informed voters. That's why our journalism is free for everyone, even though other newsrooms retreat behind expensive paywalls.
Our journalists will continue to cover the twists and turns during this historic presidential election. With your help, we'll bring you hard-hitting investigations, well-researched analysis and timely takes you can't find elsewhere. Reporting in this current political climate is a responsibility we do not take lightly, and we thank you for your support.
Contribute as little as $2 to keep our news free for all.
Can't afford to donate? Support HuffPost by creating a free account and log in while you read.
Dear HuffPost Reader
Thank you for your past contribution to HuffPost. We are sincerely grateful for readers like you who help us ensure that we can keep our journalism free for everyone.
The stakes are high this year, and our 2024 coverage could use continued support. Would you consider becoming a regular HuffPost contributor?
Dear HuffPost Reader
Thank you for your past contribution to HuffPost. We are sincerely grateful for readers like you who help us ensure that we can keep our journalism free for everyone.
The stakes are high this year, and our 2024 coverage could use continued support. If circumstances have changed since you last contributed, we hope you'll consider contributing to HuffPost once more.
Support HuffPostAlready contributed? Log in to hide these messages.