John Tolva and Brett Goldstein are leading us into a Brave New World. John is the newly appointed chief technology officer for the City of Chicago and Brett, the city's chief data officer. Together, they are turning the promise of Big Data into reality in one of the most challenging big city environments imaginable. Last week, at a meeting hosted by the MacArthur Foundation, who is helping support the city's work, I walked away with the following insights about how civic leaders can approach Big Data as a means to understand cities better, make better decisions, and improve the quality of life for more citizens.
Doing Anything With a Lot. In this early stage of innovating with Big Data in cities, the challenge is not how to do something with nothing but how to do anything with a lot. Chicago has an extraordinary amount of data but they really don't know what that data is or how to corral it across agencies. John Tolva spoke about the importance of harnessing data to do 3 types of analysis: historical (what happened before), spatial (especially cross-agency to determine what's happening in specific locations) and predictive (how can we predict, and avoid, unwanted outcomes).
Finding Patterns in All the Noise. Of course, there is a lot of interest in 'predictive analytics' or the possibility of finding patterns in 311, 911 and other municipal, large data sets in order to predict everything from health, crime and economic growth. Brett Goldstein spoke of the work he did while at the Chicago Police Department where they collected data and built algorithms so they could predict incidents within 48 hours. These city officials also emphasized the power of locational or spatial data. By collecting a lot of different data about a single location, your ability to do real time situational analysis is extraordinary.
Prioritizing the Work. In this early phase of work, the office's priorities are falling into three categories: (1) Define what the data is -- leaders are looking across many government agencies to inventory what they have; (2) Help change the mix of skills in the mayor's office -- The city needs to bring in talent that has the technical capacity and fearlessness needed for the work, but also to retrain policy analysts so that they can use the data for governing; and (3) Show results quickly especially in neighborhoods -- Use and relevance are the most likely ways that the city will overcome resistance to openness and skepticism.
Mixing Social Science and Data Science. A real advantage of working with Big Data is its mix of 20th century social science with 21st century computer science. Brett Goldstein called it bringing machine learning and tabular data together. The University of Chicago, who was at the meeting, has recently established a degree to do just that.
Disrupting Long Held Assumptions. Ultimately, the success of Chicago's efforts, and likely those of cities throughout the country, will depend on the willingness of people like Brett Goldstein to reject conventional wisdom that this can't be done in order to custom build solutions with gum and spit. This type of work will likely challenge many entrenched ideas and beliefs, and as such requires leadership, support and validation from those at the top, including Mayor Rahm Emanuel.
Alex Howard of O'Reilly Media, who also spoke before the Living Cities board recently, articulated a great formula for working with Big Data in this Brave New World: use the wisdom of crowds, power of algorithms and intuition of experts.