Big data allows startups, companies, governments, and societies to gather unprecedented amounts of insights that could change the world for good. Recently, 500 Startups invested in Social Cops, which processes millions of discrete data points in the most remote locations in the world for industry and social sector leaders to make data-driven decisions.
Startups Lyft and Spotify have used big data to revolutionize the transportation and music industries, respectively. Glow - a startup - raised $17M to further develop apps that leverage big data to empower women to gain better insights about their reproductive systems. The Climate Corporation is working to help numerous industries strategically combat and cope with the weather's unpredictable behavior by utilizing real-time computing power to simulate 10 trillion data points that monitor conditions on a global scale each and every day.
BigData-Startups Founder Mark Van Rijmenam is developing a one-stop-shop to spur the understanding and application of big data by organizations.
"Big data requires a shared understanding. It means different things for different industries, and what the outcome will be," says Rijmenam. "It's not the volume of data but delving into meanings and gathering insights. Companies don't necessarily need terabytes of data. They just need to understand how to draw insights and apply them to their business."
Kaggle -- the world's largest community of data scientists -- exists to empower the world's biggest companies to use cutting-edge data science to solve real-world problems; and, DataKind, a community of data scientists, is working to solve the most complex and intractable problems we face.
Ironically, most of the projects at MIT's Global Founder's Skills Accelerator 2014 Demo Day centered on using big data in different niche-areas to help vulnerable populations. "I was impressed," said MIT Sloan professor Edward Roberts, the founder of the Martin Trust Center for MIT Entrepreneurship and a mentor to many of MIT Sloan's entrepreneurial alumni. "These teams are the best of the best among 150 applicants and big data was ever-present. That heavily reflects the role that data analysis is going to play in the future."In his talk at TEDSalon, Kenneth Cukier said:
"Big Data is an extremely important tool that will help society advance. Big data is new. The only way this planet is going to deal with it's global challenges--to feed people, supply them with medical care, supply them with energy, electricity, and to make sure they are not burnt to a crisp with because of global warming is because of the effective use of data."
IBM's big data study, "Smarter Health care in Canada: Redefining Value and Success," collected millions of data elements from monitors in ICUs--in aggregate form, the data identified early warning signs of potential infections that newborns may have. These early warning signs are not insights even an experienced doctor would catch using traditional practices. The collection of large quantities of data will enable modeling of social systems at scales, both micro and macro, that uncover new patterns and relationships that were previously invisible.
In a recent Stanford Social Innovation Review webcast, I learned that Measure Measurement, a social impact consulting firm, is using big data to predict social impact program outcomes by creating an efficacy score based on meta-analysis research for direct service nonprofit programs. "The use of big data has the ability to change the landscape of social welfare," says Foster Skills President Eddie Vaisman. "By aggregating social welfare data, we can determine exactly what strategies are able to help vulnerable populations."
Government agencies are already modeling micro- and macro-level phenomena to improve the world. For example, The Centers for Medicare and Medicaid Services have begun using predictive analytics software to detect potential fraudulent reimbursements before they are paid out. The Defense Advanced Research Projects (DARPA) created a program, Nexus 7, that fused satellite and surveillance data to visualize traffic and road network patterns, which made it easier to find and defuse improvised explosive devices (IEDs) in Afghanistan and to save the lives of American soldiers.
The United Nations' Secretary Ban Ki-Moon also launched The Sustainable Development Solutions Network (SDSN) to accelerate problem-solving for sustainable progress in developing countries. He hopes to use big data tools when tracking the indicators of our future Sustainable Development Goals (SDGs). Moreover, University of Pennsylvania Professor Richard Berk is using algorithmic technologies to predict the likelihood of someone to commit a crime, which is both daunting and fascinating due to the groundbreaking potential this could have in the future. Predictive analysis has the ability to fuel new developments and innovations in education, healthcare, agriculture and energy use, as well as strengthen national security.
Big data, in my opinion, can be compared to a natural resource. It is a sexy new technological phenomenon that is worth investment because it has the ability to change the world when used for good.