05/04/2011 09:35 am ET Updated Dec 06, 2017

Sleep, Friends, Work -- All Victims of Data Overload

At first, I thought it was just me.

For the first time in my life, I was dropping the ball. Missing emails. Notes from friends on Facebook. DM's on Twitter. And I'm no slouch. I start early, am on the web all day, and answer my last email just before I turn out the lights to go to bed.

So, what was I doing wrong? And, was I alone?

Now, I know the answers -- and they're stunning.

After weeks of survey research, it turns out -- the bounty and abundance of web data is out of control. As Google's Eric Schmidt has been quoted, from the beginning of time to 2003, we created 5 exabytes of data. We're now creating that every two days -- and it's accelerating. Think of it like Moore's law -- for content. But unlike increased processor power, massive growth in unfiltered and un-contextualized content isn't a boon. It's a data deluge that's drowning us all.

The "Digital Lifestyle Information Survey 2011" asked respondents about how they receive, manage, and organize information -- and how it's impacting their lives.

A staggering 72.7 percent described their data stream as: "A roaring river," "a flood," or "a massive tidal wave." And 48.5 percent said that they where connected to the web: "from the moment I wake up until the moment I go to bed."
And the impact is more than just at work, people's personal lives are impacted as well. Personal time and work time have blurred, so much so that even the middle of the night is no longer of limits.

Perhaps most startling, 33 percent of respondents said they check emails in the middle of the night.

If there was a consensus, it was that the volume of data is forcing people to make painful choices between work and friends and family. 46.9 percent said "I am unable to answer all my email", and 41.4 percent of respondents said "I feel I miss important news." The personal impact was dramatic as well, with 39.9 percent saying: "I feel I ignore family or friends."

But most telling -- 62.5 percent seemed to hunger for a new solution, saying "I wish I could filter out the flood of data."

"What do you do to cope?" 79.5 percent said "I check my email all of the time." 76.6 percent responded: "I work weekends/evenings." And more than a third either gave up, or gave up sleep. 46.6 percent said "I admit I can't keep up" and 33.8 percent responded: "I try and sleep less."


It seems people have reached their capacity to manage data, impacting family, friends, productivity, and even sleep. Algorithmic solutions will in fact expand the problem, creating more undifferentiated data.

The future is not in better computers, or smarter algorithms, but in embracing the powerful human ability to find, sort, and contextualize information and ideas. It is the dawn of a new era for the web, the human filtered web. The curated web.

Survey respondents seemed to see their human connections and their best hope to stem the tide of data overload. People are engaged in 'social curation,' and see content publishing, linking, and re-tweeting as a key part of their emerging 'digital identity.' Individuals managing and sharing content no longer consider it a hobby, but part of how they define themselves in the digital and social web world.

The majority of respondents, 61.3 percent, said "I consider the content I share part of who I am." And 47.4 percent said "My followers on Twitter read my tweets and RT's for info." And 38.7 percent said "my co-workers follow me, and read my posts and tweets."

Human data management, shared and community filtering, and personal recommendations will fulfill individual's 'digital identity' as content curators -- while allowing content consumers to 'surf' less, and consume curated content delivered to them by trusted sources.

Increasingly "you are what you curate."

You can download the entire Digital Lifestyle Survey Here

(originally published on Fast Company)