Let My Data Go: How Activists Can Transform Government Through Public Data

Whoever wins in November faces record voter skepticism about whether "government is on the right track," while record deficits will make it difficult to add new programs that might rebuild public confidence. The new administration must simply get more from less.

One novel solution, which I'll address Friday in my speech at the Netroots Nation Conference in Austin, lies in an approach pioneered by the District of Columbia. It combines free access to governmental data and new Web 2.0 tools allowing anyone to turn data into eye-popping graphics that help make the information understandable.

The resulting "data visualizations" can highlight inefficiencies, break down barriers between programs and agencies, and (potentially) leverage "the wisdom of crowds" both within agencies and among the general public to find creative new approaches.

This public data movement has two components.

The first is liberating reams of government data, so more people, both inside agencies and outside, can interpret it. The Obama campaign, in its technology white paper, makes this a priority as part of its transparency goals: "... to allow citizens to make use of that data to comment, derive value, and take action in their own communities."

Several federal and state agencies release data in easy-to-use formats such as RSS or geospatial ones easily mashed up with Google Maps or combined in new ways. None are as diverse as the 216 the District of Columbia's "Citywide Data Warehouse" releases: from building permits to crime to potholes. Even better, they're released on a real-time basis, so they can actually be used to manage situations now, rather than simply analyzing historical practices.

A public data program's second component is tools to display data visually, rather than in mind-numbing charts. As Edward Tufte, the leading data visualization expert explains, "Often the most effective way to describe, explore and summarize a set of numbers -- even a very large set -- is to look at pictures of those numbers."

New data visualization sites such as IBM's Many Eyes and Swivel make it simple to produce interesting graphics, and, equally important, to share and discuss them with Web 2.0 tools such as tags, topic hubs, and threaded discussions. Already, individuals worldwide and groups such as the Sunlight Foundation use these sites to illustrate and debate issues ranging from US government spending to how nations rank regarding privacy safeguards.

Individual activists have seized the opportunity to highlight issues through data visualizations. One of the best pioneering examples is Adrian Holvotny's ChicagoCrime (which has been so successful that he's expanded its scope into a comprehensive source of neighborhood news and data: EveryBlock Chicago), although he's handicapped by the fact that Chicago delays publication of crime stats for a week after they happen, compared to DC, which publishes this data as it is recorded.

Another is Neighborhood Knowledge Los Angeles, a collaboration between UCLA and community activists, combines and plots an a single map data on 7 "problem indicators" such as code violations or delinquent property that previously remained isolated in various agencies' files. Seeing so many problem indicators on a single block's map should be a red flag to city officials to intervene NOW with coordinated services to halt the decline.

The presidential campaign offers a perfect opportunity -- quite literally now -- for activists to learn how to work with these data sources and data visualization so that their arguments and position paper can become more fact-based. The Obama campaign will hold a series of public meetings nationwide next week ("Listening to America") to help draft the Democratic Platform, and in my Netroots Nation speech, I'll issue a challenge to the attendees: "Let's make the 'Listening to America' discussions substantive, and fact based, and show that we're on board with Obama's commitment to make data available and to use it wisely."

I got the ball rolling by creating a topic hub, "Obama platform" on the Many Eyes site. The URL is tinyurl.com/5680bs . I've "seeded" it with a variety of data, on issues ranging from global warming to the housing crisis, plus some visualizations various individuals have created of those data, as well as their interpretations of the data.

I will tell the Netroots attendees: "When you leave this conference, find statistics relating to your pet issues, and upload them to the 'Obama platform' hub. Get your friends to do the same, and use the discussion features on the site to begin debate now. When you go to the 'Listening to America' events next week, cite these data and the interpretations."

The issue of data feeds and visualizations is much more than making pretty pictures: if more government agencies begin to release data and we find imaginative ways to use them to illustrate critical issues and potential solutions, the potential benefits include:

-- more informed policy debate, grounded in fact, rather than rhetoric
-- greater transparency and less corruption
-- optimizing program efficiency and reducing costs:
-- new perspectives, especially when "the wisdom of crowds" emerges.

Who would believe dry data plus Web 2.0 magic could become the engine to involve the public in governmental transformation?

Comments

[Shortyfuse]

Yes , Yes ,Yes to both the article and the previous post. And please demand release of Bush papers that he stored in a vault at his fathers library at Texas A&M.
We must get back to a figure head president and an open government both national and also the governments of the states. In Texas the republican senate passed a law so that you cannot access the voting record of your so called elected officals. This is important since the just passed 600 NEW LAWS. 600 NEW LAWS FOLKS.

[phasechange]

Hear, hear! Let's take things a step further. To make government data easier to use and more powerful, data need to be designed and collected in a way to make it easier to search, retrieve, and integrate. For example, if there were a "Common Corporate ID" number used by every federal agency to identify the corporations they dealt with, anyone, either citizens or government enforcement lawyers, could develop a federal-wide profile of that corporation. For example, if a company submitted a permit application to the EPA, the permit review team could gather data on that company's activities before Agriculture, Interior, and so forth. Such an ID number already exists - the Federal Employer Identification Number (EIN) issued by the IRS. Congress should mandate that this number be used whenever any company submits data to a federal agency.

Another similar number would be a "Federal Facility ID". Every physical industrial operation, such as a refinery, would be assigned a unique number. As with the corporate ID, data across federal agencies could be much more easily integrated.

As proven by research undertaken by leading federal policy groups, such numbers would also save money. Corporations who report to the federal government would have less reporting burden since they would only have to provide data once and it could be shared across all federal agencies, and the state and local governments as well. The data analysis burden would also be reduced for federal employees, public interest advocates, and industry groups as well.