You know it.
As we speak, big data is transforming businesses and our very lives.
But exactly how? What are the key trends in big data that are changing how we work?
Which of these trends are your competitors leveraging to get an edge in the industry?
We asked one question to 38 well known experts in big data,
"What's one current trend in big data analytics that you are excited about?"
Their answers will give you valuable insights into the world of big data.
Over to our guests (in alphabetical order)
CEO | Basho Technologies
"We're at a point where companies can finally have the opportunity to do some exciting things with IoT data. As companies invest in their IoT strategies, they're essentially making multimillion-dollar bets about where the IoT and data analytics industry is going. I believe edge analytics initiatives are a definitively good bet for enterprises; they will allow companies to derive the most actionable value from their data. It isn't feasible to move petabytes of data that IoT creates from the source all the way to a centralized warehouse or analytics location. It's too expensive and too time-intensive. Edge analytics is the next step for companies that want to gain real-time insights and impact business through IoT use cases."
CEO | Trifacta
"Companies with big data initiatives are waking up to the fact that they can't achieve reasonable ROI from their data lakes if they only allow a handful of highly technical users explore the data. Companies have to find ways to democratize access to the information on a massive scale, allowing a much broader set of data analysts and business users work with the data directly. To support these new users and their analysis, an entire ecosystem of self-service tooling is emerging, including self-service analytics and self-service data preparation. These emerging tool chains will increase information agility, allowing the people who know the data best to take it from raw to refined and onto analysis in a fraction of the time; delivering more projects faster and driving better decisions everywhere.
CEO | Sisense
"The democratization of Analytics is accelerating even faster these days. As the classic BI value chain is breaking down and IT is giving/losing control to the business, a new class of business data heroes emerge. A degree in Data Sciences and IT is no longer needed - MBA is the new DBA. Business owners throughout the modern enterprise are looking for tools that would allow them to ingest terabytes of data from a broad range of disparate data sources without the need to chase their IT department for a report that would arrive in 2 weeks, maybe. Predicting the past is no longer an option - those days are past, the modern business needs answers to complex data questions today. "
Chief Scientist | Booz Allen Hamilton
Former CEO, Cognitro Analytics
"With advancement in big data computing, machine learning and IOT, technology is no longer a hard problem in the game of analytics, governance is !!! To govern with data requires that you first govern the data, that is, bringing the right data products to the right people at the right time and within the right legal and ethical framework"
President & Founder | Mammoth data
"Mammoth Data has been moved to focus increasingly on helping customers with key Data Science problems as customers move to the cloud and towards more operationally mature Hadoop infrastructure. Really getting past the infrastructure and focusing on what looks like magic and is really just math and processing data through memory in real-time excites me."
CEO & Founder | Interana
"One trend we're excited about is the continuing shift toward raw data. When Big Data first caught on, everybody got excited. Then they ran into the limitations of their technology. There was just too much data to handle raw, so they started sampling at the collection points or aggregating their raw data streams. Sampling and aggregation reduced the volume of data to a something that could be analyzed without breaking the bank. It was a smart and pragmatic strategy at the time, but it's especially limited when it comes to behavioral analytics. When you want to understand behavior, all the raw event data might be relevant. Dropping or aggregating away individual actions skews funnel and path analysis, forces sessions to be rigidly defined, and makes it impossible to understand the actions of individual actors. There's still an important role for sampling, but it comes after collecting the raw data and must be tailored for behavioral analytics. As more organizations recognize the critical value of behavioral analytics, we're seeing a shift toward collecting and analyzing raw data. Interana is a purpose-built solution for behavioral analytics of event data at massive scale. The solution consists of a highly scalable cluster which is combined with an intuitive visual interface to interactively explore trillions of events in seconds."
Founder & CTO | simMachines
"Machine learning is all about predictions. It tells you what is going to happen. But it cannot tell you WHY is going to happen.
If you know something is going to happen but you don't know WHY is going to happen you cannot change your fate. We believe that the WHY is a critical component in Machine Learning and new technologies are being released in the market that address this specific and fundamental problem."
CEO | Datastax
"The current trend that excites me the most is the realization that we must turn 'data' into actionable 'information' within the timeframe of interactive transactions. Seeing how a customer experience can change within a split second is truly revolutionary for every aspect of our lives including medical, environmental, security, entertainment... you name it."
CEO | Couchbase
"One trend that is going to continue to pick up steam in the coming year is the integration of operational NoSQL databases with analytical platforms like Spark. Web, IoT and mobile applications generate massive amounts of data that has very little intrinsic value for a business. To effectively derive real value from this raw data, it's important for companies to pair their analytical platform that rapidly extracts insights from the data with their operational database that can quickly turn those insights into actions to improve customer experiences and operational efficiencies. Ultimately, shortening the time-to-insight and time-to-action is a competitive advantage that enables companies to find the true value hidden inside their data."
CEO | Snowflake Computing
"The key trend in big data is the transition of analytics solutions into the cloud. For data analysis, this is huge breakthrough. Of course, the cloud enables vast amounts of computing resources to be applied to data analysis and to scale that computing based on need.
But more importantly, the transition to the cloud enables organizations to acquire this technology as a service, fully offloading the burden of building these complex solutions onto companies who have made this their business. Traditional big data solutions are hugely complex and require specialized skills that are unavailable to most companies. This complexity has prevented broad adoption of big data technology.
Providing data analysis as a cloud service creates the opening for all companies to gain insight from their data. This is the breakthrough that will enable the rapid adoption of big data technology across globe."
Founder & CEO | Trinity Insight
"Truly customized digital experiences. With the advent of bid data and the convergence of call center, CRM, and ecommerce platforms - consumers will get truly personalized experiences on their devices.
Brands will be able to message and merchandise online based upon in-store and online purchase data in combination with clickstream data. If a consumer bought a hockey goal today in-store and that purchase is associated to an identifier such as an email address, consumers can provided with tailored alternative products (ex. goalie pads) and opt-in related lead magnets (ex. 10 drills for hockey teams) for nurture campaigns."
President | CogNova Technologies
I will give Two perspectives:
"Technical/ Scientific: We are coming to better understand how human's learn from current research in deep learning and lifelong machine learning. These are systems that develop rich representations of the world from data (customer attributes, images, sound) and that can be used to better learn new classification and prediction tasks over time from many related tasks .. like humans."
"Applied: Cloud-based modelling and visualization utilities are now emerging from Google, IBM, Microsoft and others that are starting to bring data analytics to even the smallest of companies and new start-ups. This will generate a lot of new predictive and prescriptive analytic business ideas. Coupled with a rapid increase in sensory data from the Internet of Things, this will make for exciting times ahead. Imagine your smart watch alerting you to your lack of movement over the last 30 minutes, providing a short Fitbit report, and suggesting a prescription of office activities to make you more productive and healthy."
CEO & Founder | Chartio
"Democratized data exploration is the most exciting trend we'll see over the next decade. We've gone through the big data phase of storing tons of data and now have quite a few standard and scalable solutions for doing so. The next step is exploring that data, and democratizing access to it. Right now engineers and analysts are the only ones with the skills and tools to drive businesses and decisions with data. Over the next decade the tools will become better and skill sets will increase such that anyone in a company will be able to utilize data."
Chairman and CEO | Alteryx
"Most important trend is the emergence of the self service data analytics worker. These newfound 'citizen data scientists' are leveraging big data, little data, structured and unstructured data using powerful and easy to use platforms like Alteryx. They are loving their jobs again."
Founder | Predictive Analytics World
Author, "Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die - Revised and Updated"
"The most actionable win from big data is predictive analytics, since each of the millions of per-individual predictions it generates directly inform the treatment or action taken towards that individual - such as whether to call, mail, approve, test, diagnose, warn, investigate, incarcerate, set up on a date, or medicate. This is the form of information technology that's transforming all the main activities organizations do, bolstering the effectiveness of our largest-scale operations.
"Predictive analytics applies across sectors and functions - it targets marketing, streamlines manufacturing, drives fraud prevention, improves financial decisions, optimizes social networks, empowers spam filters, fine-tunes law enforcement investigations, improves healthcare decisions, and optimizes political campaign activities."
CEO | Big Data Scoring
"We launched our business more than 3 years ago and back then "big data analytics" was still pretty much a buzzword only hardcore tecchies knew and used. This meant that when we approached our potential clients and offered them our big data credit scoring solutions, we almost always had to start the discussions from big data ABC. Nowadaws, the buzz has faded a bit and has been replaced with actual working big data solutions. Also, as people have had time to read about big data, we don't have to educate the market ourselves so much any more. Today, we rarely have to start meetings with answering the question "What is big data?". Today, we can jump right to our offering and people are well educated to understand and ask followup questions.
So in a nutshell... being in the business of selling big data solutions, we're excited to see big data based solutions becoming something real and tangible companies are actually using an looking into."
CEO | Panorama
"After several years in which the focus was storage of big data material, the focus has shifted to analytics and insight finding. Organizations have spent significant resources to acquire large amounts of data, but now they find that the old tools to analyze data don't apply. It is now up to us (the technology developers) to invent new concepts and tools to serve this new age"
Co-founder & President | CoolaData
"There's an exciting new breed of BI, it's called Behavioral Analytics. Behavioral analysis goes deeper to answer the more complex questions like, How do users behave over time? and Why do they behave this way? What is their next predicted behavior? or How to reduce churn? Or better, How to encourage retention? Behavioral analysis that leads to data driven insights about user acquisition, churn prediction, drive retention or LTV optimization.
Behavioral analytics relies on big data; events collected from all touchpoints and consolidated with multiple data sources. Non-sampled, unstructured, free from modeling and pre-planning. That's the nextGen of analytics - dynamic, powerful and deep. The kind of analytics any online or IoT company that depends on user behavior for business growth should adopt CoolaData offers the shortest path to behavioral analytics. A cloud-based complete solution that includes all infrastructure components for data tracking, warehousing, ETL and data enrichment through to advanced visualization."
CEO | Profitect
"The big data/analytics space is hot and evolving rapidly but prescriptive analytics is what will truly change the way we do business. The ability for technology to help organizations dynamically move beyond traditional business intelligence (BI), turning data into actionable insights in plain language, translates into immediate results - and increases the top & bottom line. Companies no longer have time to wait months or even years & relying on aggregated dashboards to identify critical business issues, figure out a resolution and act on it. Prescriptive analytics can do in weeks what some organizations have been trying to do for decades and that is what is most exciting!"
Cofounder & Exec. Chairman | Databricks
"As organizations everywhere are collecting more and more data, extracting value from this data is becoming more and more challenging This is due to both the growing heterogeneity of the data, and the growing demands from organizations to get deeper insights and build sophisticated data applications to maximize business value and ROI.
To address these challenges, there is a growing need for a unified data platform where users can do anything from data ingestion and wrangling, exploration, advanced analytics, and building data applications. Such a platform obviates the need for managing and learning different data processing and analysis tools, and ultimately democratizes data access in enterprises by enabling data engineers, scientists, and analysts to easily share data and results with the rest of the organization.
Apache Spark is such a platform. Spark goes beyond batch computation, providing a unified platform that supports interactive analytics and sophisticated data processing for machine learning and graph algorithms. As a result, Spark maximizes big data project ROI while decreasing time-to-value and reducing the cost."
Founder & CEO | Statistics Solutions, LLC
"Big data is exciting because it takes the world from describing events to predicting events in very nuanced ways. But the really big news, and the software products that we're developing at Statistics Solutions called Intellectus Statistics, is taking statistical analyses and models, making them incredibly easy to conduct, having them interpreted, and then written in plain English with intuitive tables and figures. Essentially, the software lets big data be accessible to the masses.
An academic version of Intellectus Statistics that we're just completing let's social science students conduct a wide range of statistical analyses, and in seconds have a Word document that students can read and understand. Statistical analysis has always been a thorn in students' sides, but we believe that our technology will transform the way students learn and will disrupt the accessibility of statistics globally. In every vertical market an Intellectus Statistics will be available--and that is very exciting!"
CEO & President| Platfora
"We're seeing a maturation of the big data analytics space through tools that can simplify processes, driving a lot of new possibilities. For example, we're seeing a democratization of data that is empowering a new generation of what Gartner calls "Citizen Data Scientists." Effectively, these are really smart business analysts--power BI users who can benefit from access to organizational data by pulling their own reports, driving their own low-level insights and easing the burden on enterprise data science teams.
Data scientists still have a very important role to play in a company's Big Data initiatives, but the rise of these citizen data scientists will enable them to focus on larger breakthroughs through data analysis, and less on what often amounts to data janitorial work. Most importantly, by finding a way to ease the bottleneck around data science teams, CIOs are finally starting to see ROI on big data."
CEO & President | ClearDB Inc.
"The most exciting development in big data analytics is the sea-change it's driving in the Database market. Businesses with requirements around big data analytics are deploying Cloud and hybrid alternatives to overcome the complexity and cost of traditional database solutions that were never designed to support a digital economy. The integration of databases into the infrastructure services stack is accelerating business innovation. Businesses no longer need a proprietary solution to unlock real-time business insights."
Founder | Elder Research, Inc.
"The biggest problem in analytics is what I call the "vast search effect". Powerful algorithms generate billions of hypotheses to test. This is great for finding previously unthough-of relationships, but on the flip side, the apparent best result could be a spurious correlation - a pattern that doesn't hold up out-of-sample. (And working on new data is all that matters!)
During two decades of consulting in data mining/ data science, we had developed useful ways of guarding against this, but the need to define a solution clearly, and disseminate it widely became urgent when I read the fascinating (and unnerving) cover article of the Oct 19 2013 Economist "How Science Goes Wrong". There, they revealed the crisis of unreliable and un-reproduceable results -- particularly in Epidemiology, where possible causes of medical problems are teased out of "opportunistic" data, where the luxury of a designed experiment is not possible. Half of the problems described had to do with the business and politics of science - how apparently positive results are more publishable than negative ones, how verification efforts don't get the respect they deserve, etc. But half of the problems were due to bad data analysis; and that half, we could solve.
Of course, it would be extremely useful if the scientists who are experts in psychology or biology etc. teamed up to write papers with a scientist expert in data analysis. But, short of that, could we share a better way to evaluate significance than the formulas derived by statistical experts a century ago when it was much more common to be testing a single hypothesis, not a swarm of machine-learning-generated ones? The answer is yes! I began to speak widely of a technique I call "Target Shuffling" which provides a distribution of best apparent results against which to compare your finding. A colleague and I explain this procedure, step-by-step, on a somewhat famous false finding: "Orange Cars Aren't Lemons?." With it, you can tell how likely a false result could arise, on your data using your algorithms. I'm excited because the realization that a solution is at hand is growing, and its use will make everyone's results more reliable. This is crucial if Data Science is to avoid the almost-inevitable backlash from over-inflated expectations now that it's gotten a huge burst of interest."
Founder & CEO | DataSelf Corp
"Mid-sized companies are slowly engaging with ERP and CRM systems in the cloud. It's still a small fraction of the market, but the trend is promising. I'm very excited that this trend will enable easy and affordable big data analytics for them. After all, we're talking about half a million companies with 20 to 500 employees in the US alone. It's a huge market!"
Founder & CEO | Zoomdata
"We see more organizations wanting to build data-driven applications that can access virtually any data source with sub-second response at scales of up to billions of records. With contextual analytics, these applications easily outperform standalone BI tools -- reducing time to insight and delivering analytics at the speed of thought."
President & CEO | Insight
"We recently surveyed over 400 IT professionals with decision-making responsibilities, and our Insight Intelligent Technology Index found that mining big data for business intelligence has already had a profound effect on IT - and business broadly - with 61 percent of respondents saying it has already impacted IT and 21 percent saying it will in the next 1 - 2 years. The first step on big data was to figure out how best to capture it, and then the focus shifted to how best to analyze it. Now, it's about leveraging the best tools to display it in a meaningful and user-friendly way, which is why we found that 54 percent of respondents said that dashboard and data visualization applications is the top new technology they are budgeting for in 2016."
CEO | QlikTech
"We're in a time when data is exploding around us. More and more people are coming to a realization that if we can turn that into insightful information, not just for a few experts but for the broad employee and user base in an organization, there is tremendous benefit. Make it very simple for the user to get going and make changes to the applications. This consumerization of enterprise software today is common; if it doesn't work on a smartphone, if it isn't super intuitive, it's simply not going to be used at all.
You have millennials coming into the workforce with the expectation that big data analytics software works like anything else on their smartphones. Another trend is collaboration, analyzing data in a workgroup. Big data analytics is not standalone tool anymore, it gets embedded in other environments. This is not a typical IT type of deployment where only a few people have the skills to deal with it. There's another trend with packaged data as a service. What if you could be approached through software with a link to start acquire curated, normalized data that is readily available instead of spending time searching for it? It could be benchmark data from an industry, socioeconomic and macroeconomic data.
Qlik's strength is that we want to give you the whole data set. The initial thing that spurred your interest, or the question that drove you to look into the data is not where you're going to end up. You're going to ask more questions depending on what the initial findings show you, the question you couldn't formulate in your head before you started looking into the data set."
CEO/ Chief Data Scientist | KNOYD
"Automation and adaptive learning. I think models requiring very little maintenance are the future, if we want to make Data Science really mainstream."
Co-founder, CEO | CIRRO Inc.
"IoT is a new frontier for big data analytics. I am particularly excited about Industrial IoT where I see a number of opportunities to improve manufacturing, downtime, operations, product quality and the total cost of ownership of high value assets. Beyond the increases in data volumes and velocity, IoT provides us with a number of new data challenges which include edge analytics, integration of smart sensors and devices, integration of IT and OT data and new requirements for data quality. Indeed, IoT will require us to rethink the ways we move, store, manage, govern and harness the value of data."
CEO | Gainsight
"More sophistication in presenting data to business users so they understand (1) what caused the model / score to show up as such and (2) what to do about it / what action to take."
President and Founder | Evolutionary Systems
"Innovation! With the exponential growth in big data, people are engaging in developing innovative procceses to find and use the data. Most important is we are seeing more collaboration between end users and analysts as well as across organizations and silos of interests. Putting the data to real use where it is of value globally is exciting. "
Partner | Boire Filler Group
"Perhaps the most exciting development is access to data that was essentially unavailable to most data scientists up until the last several years. Yet, the challenge for data scientists is to identify what data is meaningful from exponentially increased volumes of data. Essentially, this is the focus of most discussions in Big Data analytics as determining the right information to solve a given problem remains the number one challenge for most organizations.
Data scientists today are now dealing with information that is increasingly semi-structured and unstructured. This requires new tools and skills in order to meaningfully extract the data with the end result being to increase the potential analytics capabilities of the organization. But the key to real success is not technology and data but the hybrids or practitioners who will utlimately bridge the gap between using the right information to solve the right business problem."
Senior Consultant | The Modeling Agency, LLC
"I'm excited to see that a strategic question the CEO asks today can get a data-driven answer within a few days or weeks, not months or years, if at all. This was sometimes possible in the past as well, but that ability was not recognized by the C-suite until big data analytics became popular. Only then did executives begin to direct such questions to the analytics team instead of the strategy experts in the corner office or from some big management consulting firm. What a coup for analytics! Thank you, thank you big data for making it all possible. This was my motivation to compile my experience and research on the subject into a condensed and targeted vendor-neutral seminar on the topic entitled "Big Data Rhetoric and Reality."
Founder | Alta Plana Corporation
"My own specialization is in natural language processing, so the trend that most interests me is the application of NLP -- including through the adoption of machine learning -- to challenges that involve automated language understanding and language generation that range from text analytics to intelligent assistants to deep understanding of human affective states including emotion."
President | Denologix Inc.
"I believe human race is at the very early stages of an exciting future that will allow us to filter, refine and enhance our thinking based on unprecedented amounts of facts easily accessible by everyone equally; made possible by our ability to generate, store, analyze and disseminate huge amounts of data at blinding speeds."
Chairman & CEO | Attunity
"Kafka adoption is growing quickly as enterprises embrace stream processing for Big Data, this is exciting because streaming data to and from multiple targets creates new possibilities for real-time analytics, leveraged for example to assess website traffic, make location-based retail offers or enable myriad Internet of Things use cases."
Founder & CEO | OdinText
"Customers are realizing what we've been telling them, that not all data are created equal, some data even if somewhat smaller can be far more valuable while some really useless data can be very big. The first thing we ask someone about data is what is your objective? If they can't answer that we have to stop and help them. Data big or small is second priority."
CEO | FICO
"I'm seeing two things happening with big data analytics. First, after years of talking about the promise of big data, we are finally seeing the emphasis move to where it belongs - on precision decisioning leveraging big data. The promise of improving decisioning by incorporating more data in the decisioning is finally coming to fruition. Second, the cost of incorporating data-driven decisioning in workflows is down dramatically, allowing for building precision decisioning into workflows that have historically relied on crude rules."
Which of these trends will you leverage to take the next big step in your organization?