Good decisions require good data. Toyota and NHTSA's recent struggles validate this simple principle. The problem, unfortunately, is not isolated to one automobile manufacturer -- or one drug company, or one Ponzi scheme artist. The problem is a systemic lack of quality data and an historical deterioration in the analysis of those data by Government Agencies. Industry and Government are guilty of a system of "Don't look - can't see."
Regulatory agencies suffer from a lack of sufficient data. This inhibits their ability to make appropriate decisions that balance the risks between certain devastating impact to a company and a possible shift in the public's risk-benefit. Poor data gives the upper hand to Industry. Regulators and politicians correctly err on Industry's side; the alternative jeopardizes shareholder value and jobs with certainty without an appropriate weight of evidence for an alternative decision.
This Industry-weighted imbalance is often decried as favoritism and pinned on lobbyists rather than the true culprit: bad data. We rarely hear about something so mundane as data. It is far easier, and better op-ed fodder, to imply venal acts by unscrupulous former civil servants who are employed by Industry and exploit their cozy relationships with former colleagues. These articles inherently blame the existing civil servants for listening to their former colleagues but never consider the missing link: the vacuum of reliable data.
Even the latest testimony from NHTSA in relation to Toyota bemoaned the fact that the Agency did not have enough technical expertise to properly evaluate electronics and other technically sophisticated equipment. While accurate, it is in many ways a secondary issue. Technical expertise is easy to acquire. There are more than enough universities and consulting companies that can provide state-of-the-art know-how to evaluate any system - certainly better than any single employee hired by an Agency.
The problem is data. For most problems, be it automobiles or pharmaceuticals, the failure rate is usually small, if caught early enough. But the smaller the failure rate, the more data is required to make a responsible decision. The data requirements become even more stringent as the number of alternative factors increases (e.g., patient allergies, age, general health; accidents on wet surface, night time, speed).
At any given company, one can imagine a meeting where the VP for Sales questions the validity of the data from the VP for Quality Assurance and suggests postponing an action until better data is available. Yet, any law passed that requires manufacturers to provide data will only elicit the data the manufacturers themselves have in their possession. But, through no fault of their own, this might not be enough. A drug company cannot report an adverse effect for which there is no notification. Toyota cannot report a crash for which they have not been notified.
In many instances, no matter how strong the incentive to make reasonable, proper decisions (e.g., liability, penalties, criminal charges) Industry will not find the problem because it is unable to force (or unwilling to pay) outside institutions to provide the necessary data. Unless Congress is going to expand the powers of regulatory or consumer oversight agencies to force aggressive data collection, provide broad domain for accessing Industry data and enable these agencies to collect data from alternate sources (e.g., insurance companies, hospitals, doctors, repair shops, state systems), then no matter how many people these agencies employ, problems will continue to occur.
The lack of data makes oversight agencies less effective. A perpetual lack of data has an enervating effect on regulatory agencies. What is the point of analyzing data on a routine basis if there is never enough data to make a meaningful pronouncement? Quality data spurs analysis and nurtures analytic creativity. One must question why NHTSA found the Toyota statistics "unremarkable" while State Farm Insurance saw patterns that concerned the company enough to call NHTSA.
Thus, while improving the Government's ability to collect data is mandatory. Transparency cannot occur without the establishment of data warehouses within these agencies that routinely publish reports that identify for Industry, Congress and Consumers benchmarks that help all make better decisions and are a barrier to internal and external pressures, as well as, adding impartial information to any litigation.
Some might worry about the Government collecting more data -- yet this is paranoia without historical basis in fact. The Government collects census data, social security data, Medicare data, and corporate secrets data (e.g., EPA's Office of Toxic Substances) and these data and their sources have remained safe. At the same time, insurance companies, banks, credit card companies and corporations have collected and shared all forms of consumer data with often well documented lapses in security and perhaps little benefit to the consumer.
Litigation in these matters is always about the same questions: What did you know and when did you know it? What should you have known and when should you have known it? What actions did you take and when should those actions have been taken? Any action Congress takes to enhance data collection increases the likelihood of improved decision-making and, ultimately, public safety.
[Editor's Note: An earlier version of this post incorrectly stated that Liberty Mutual Insurance was the company that reported these concerns to NHTSA. It was actually State Farm Insurance.]
The author is a PhD statistician who was an expert witness for Firestone in their Radial ATX matter. He is a former employee of the US Environmental Protection Agency and has written numerous epidemiological peer-reviewed papers.