Can Big Data Subdue Big Government?

05/14/2015 10:54 pm ET | Updated May 14, 2016

Asymmetric approaches to improving government transparency and accountability.

U.S. politics is a frustrating maze. We sense sinister forces at work to gain undue influence and advantage within it but it's so very hard to find and expose. At best, acting upon discoveries happens years after damage has been done. Often all we can do is slap wrists. It means there is presently very little to deter politicians and special interests from acting with impunity.

This is a very sad state of affairs. It leads many people to abandon participating and relinquish control over their lives. It activates others to take polarized sides seeing fellow citizens as mortal enemies. They fight each other while the true culprits who benefit from societal fratricide hide in the shadows and take advantage of the chaos. There is less and less credible threat to dissuade this sort of destructive behavior as time goes by.

But what if there was? What if, as disruptive innovation has done in so many cases, there was a way to detect suspect behavior and expose it in real time? What if indications could be back tested for patterns of misbehavior and the results of objective analysis communicated to those specifically affected so that everyone could see officialdom beginning to stray as it happens? Would such a detection and warning system force politician's hands to be cleaner? Would such a system expose patterns of action by special interests across broad swaths of action, now deftly hidden, and make them plain as day?

This is exactly what "big data" aka "deep learning" technology does. Let's say you wanted to find politicians who might be peddling influence out of their offices. One could absorb every revision of every legislative bill authored by every political office holder in every legislature for every legislative session for as far back as every government entities keeps electronic public records. In terms of big data, that's actually not a lot.

Now here's the take the power back part. Automate searching for every bill that shows it experienced a radical revision of content indicating it was altered from one intent to another by the author. Don't bother to read what's in it. The warning sign is that someone was trying to bypass the legislative process by slip streaming a change to fast track a special interest's influence ... any influence.

That's not enough to prove anything by itself. But it can act as a trigger to then cause a follow on big data algorithm to examine every piece of legislation ever sponsored by that politician in their entire career regardless of how many offices they have held to see if it's happened before. If so, how often? And, is the pattern of positions taken when these radical alteration occur consistent or random? A pattern of random ones that have no policy cohesion is actually the most worrisome that the legislator might be a reed in the wind.

The thing about big data algorithms running in real-time is they tend to be harshly objective. They don't care if your position is left or right. They just report they found an anomaly ii the matrix. That anomaly may just as easily turn out to be laudable as it could turn out to expose crookedness. But it will bring what was in the dark shadows into the bright light for all to see.

How many legislators warranting headline risk explaining do you think such a system would detect if it were running right now? How many exposures of malfeasance do you think it will take to begin change the behavior of government? Would it change incentives enough to deter future inappropriate behavior if politicians knew their careers were a very, very open living document?

That's just one data mining scenario. A proper legislative big data transparency system would run tests on other artifacts of governance. It would run additional tests to reveal patterns about legislative issues erupting on the scene in many places at once and even the extent to which special interest organizations are acting behind events nationwide.

You don't actually believe a legislator that guts a placeholder bill and replaces everything below the bill number re-wrote it in a moment of Jeffersonian brilliance do you? Neither should you believe the same bills with almost photocopied legislative points miraculously appeared in half the legislatures at the same time as the result of a lucky fluke. That's the signature of big money at work.

Note that the "big data" needed to run such and analysis is almost all public record. Technically, anyone with the equipment, software suite and programming skills can do this. What's your reaction to using big data this way? Do you think of this thought experiment about what disruptive technology could do to radically alter the playbook for government transparency and accountability is a good thing or a bad thing?

Dennis Santiago was the architect and author of the "Move Your Money" system that delivered an internet based means for individuals to find safe and sound community banks near their zip codes during the middle of the 2008-2012 financial crisis. The service was provided to the public for free at a time when the need for such transparency was needed by a very nervous nation.