Where Is All the Big Data We've Been Promised?

03/11/2015 12:19 pm ET | Updated May 11, 2015

Author Jesse Lipson is a VP & GM for Citrix, which acquired his company ShareFile in 2011. ShareFile is a secure file sync and sharing service that meets the mobility and collaboration needs of users and the security requirements of businesses. He launched ShareFile in 2005 and bootstrapped the company from zero to four million users in six years. Jesse has also launched and/or led several other cloud and e-commerce businesses, including, a pharmaceutical market research company, and the website development firm novelProjects.

When "Back to the Future 2" came out in 1989, Marty McFly's trip to 2015 got me pretty excited about the cool things we'd be doing by now. So it's pretty disappointing to arrive in 2015 and find out that I still can't commute to work in a flying car or cruise down the sidewalk on my hoverboard.

Will Big Data suffer the same fate, always being 20 years away from reality?

From Basic Needs to Self-Actualization: The Progression of Data

We've come a long way in the last 10 years toward realizing the full value of our business content. We are well on our way to not only having easy, free access to vast amounts of cloud data storage, but also to feeling as viscerally entitled to it as we do life, liberty and the pursuit of happiness. And that bodes well for the promise of Big Data.

As cloud storage becomes ubiquitous, our data needs will evolve. The next innovations will focus on better, faster ways to use, find and mobilize data. Cloud computing is no longer just a way to share and access information. It's the answer to smarter work, higher productivity, and revelatory insights that transform our lives in ways both big and small.

It's the next step in the necessary progression of the industry. Take a look at Abraham Maslow's famous Hierarchy of Needs, which posits that basic physiological needs must be met in order for human beings to progress to our highest potential. Maslow's pyramid rests on the basics for survival -- air, water, food, sex. Once those are satisfied, we can focus on safety and security. Once we're safe, we seek love and a sense of belonging. Then we look for respect and esteem. Only when all of these other needs are met can we focus our energies on self-actualization, the realization of our full potential in life.

When applied to data, the model looks like this: At the foundation is data creation, something we've done since the beginning of time. Next is storage and access to data, industries that have radically changed in the last few years as cloud computing has become the norm for business and for personal use. Next is the optimization of workflow, search and discovery -- all heading toward the goal of leveraging data for transformative insights that will change our work, and even our world, for the better.

4 Ways the Future of Big Data Will Change the Way We Work and Live

Just as we strive for self-actualization in our lives, what we ultimately want in our work are tools that help us work faster and smarter, and that help us achieve big goals. And now that we've built a solid ecosystem for the data basics, the industry will focus on innovation in these higher realms. So here's what we can expect in the coming years:

1. Less process, more flow. We already share documents quickly and easily. Next, we'll reduce the process around sharing. No more emails to share links to files; instead, workflow will be automated based on actions, dates and content. Your action on a document will trigger the next person's actions. Contracts will be signed and executed in moments, approvals generated in seconds. No matter where a file is hosted, work will be paperless, the flow of information seamless.

2. No more guesswork. Right now, your data probably lives on company networks, personal and professional file-sharing services, email, computer hard drives, mobile devices and more. Remembering where you put any particular file can be hard -- and soon you won't have to. All of these services will be connected by one, simple interface. The file you need will be served up to you, when you need it, from wherever it lives, via global search tools that learn your needs and preferences to become even more useful over time.

3. Meet your new personal assistant. Once the simple search problems are solved, machine learning will help us move up the hierarchy of data needs by delivering relevant content, in the right context, at the right time. Services will proactively recommend content based on your work and your actions. A social media-type feed could surface recent work from colleagues that relates to your own projects. A calendar system could automatically populate your meeting invites with relevant documents and emails. Our devices will seem more like reliable, intuitive friends than computers -- and the movie "Her" will suddenly seem much more plausible.

4. Big data comes to the rest of us. Already we're seeing tantalizing hints of how data can transform our lives. It's helping to improve disaster recovery through improved communications and mobilization of resources. It's leading to new insights in education. It holds promise, even, for curing cancer. As data becomes more transparent and accessible, we'll all have opportunities to leverage aggregated knowledge and information in ways that will help us work smarter and solve bigger challenges.

Some of these innovations have been promised for years, but attempts have failed because the groundwork wasn't done. There's no hoverboard that can help us skip over the hard work of building the foundation for the future. But the exciting future of Big Data will come. We just need to master the basics first.