THE BLOG
03/22/2013 11:42 am ET Updated May 22, 2013

The Current State of the Data Union

The following is excerpted from "Too Big to Ignore: The Business Case for Big Data" (John Wiley & Sons, now available).

Unstructured data is more prevalent and bigger than ever. This does not change the fact that relatively few organizations have done very much with it. For the most part, organizations lamentably have turned a deaf ear to this kind of information -- and continue to do so to this day. They have essentially ignored the gigantic amounts of unstructured or semi‐structured data now generated by always‐connected consumers and citizens. They treat data as a four‐letter word.

Too Big to Ignore

It's not hard to understand this reluctance. To this day, many or­ganizations struggle managing just their own transactional and struc­tured data. Specific problems include the lack of master data, poor data quality and integrity, and no semblance of data governance. Far too many employees operate in a vacuum; they don't consider the impli­cations of their actions on others, especially with regard to information management (IM). Creating business rules and running audit reports can only do so much. Based upon my nearly 15 years of working in different IM capacities across the globe, I'd categorize most organi­zations' related efforts as average to poor.

For every organization currently managing its data very well, many more are doing a poor job. Call it data dysfunction, and I'm far from the only one who has noticed this disturbing fact. As for why this is the case, the reasons vary, but I asked my friend Tony Fisher for his take on the matter. Fisher is the founder of DataFlux and the author of The Data Asset: How Smart Companies Govern Their Data for Business Suc­cess. Fisher told me:

The problem with data management in most organizations today is that they manage their data to support the needs of a specific application. While this may be beneficial in the context of any one application, it falls woefully short in supporting the needs of the entire enterprise. With more sources of data -- and more variety of data and larger vol­umes of data, organizations will continue to struggle if they don't adopt a more contemporary and holistic mind‐set. They need to reorient themselves, aligning their data man­agement practices with an organizational strategy instead of an application strategy.

In other words, each department in an organization tends to em­phasize its own data management and application needs. While this may seem to make sense for each department, on a broader level, this approach ultimately results in a great deal of organizational dysfunc­tion. Many employees, departments, teams, groups, and organizations operate at a suboptimal level. Sadly, they often make routine and even strategic decisions without complete or accurate information. For in­stance, how can a VP of Sales make accurate sales forecasts when his organization lacks accurate master customer data? How can an HR Director make optimal recruiting decisions without knowing where her company's best and brightest come from? They can't -- at least easily.

Far too many organizations struggle just trying to manage their structured data. (These are the ones too busy to even dabble with the other kinds of data discussed earlier in his chapter.) The results can be seen at individual employee levels. Because of poor data management, many employees continue to spend far too much time attempting to answer what should be relatively simple and straightforward business questions. Examples include:
  • HR: How many employees work here? Which skills do our em­ployees have? Which ones are they lacking?
  • Payroll: Are employees being paid accurately?
  • Finance and Accounting: Which departments are exceeding their budgets?
  • Sales: How many products do we sell? Which ones are selling better than others? Which customers are buying from us? How many customers in New York have bought from us in the past year?
  • Supply Chain: What are our current inventory levels of key products and parts? When can we expect them to be replen­ished? Will we have enough inventory to meet current and fu­ture demand?
  • Marketing: What's our company's market share? How has that changed from last year or last quarter?
For organizations with cultures that have embraced information‐based decision‐making, answers to questions like these can be derived quickly.

To order the book on Amazon, click here.

Subscribe to the Culture Shift email.
Get your weekly dose of books, film and culture.