12/10/2013 08:57 am ET Updated Feb 09, 2014

Management Science Is a Delusion

Have you ever stopped to think how much energy and money goes into teaching management science? How much time is consumed by talented teachers and learners? Or how much effort well-run companies put into their procedures and decision-making? All those analysts, all those spreadsheets, all those investigations and calculations? All the business schools, university management courses, institutes of business, and distance learning courses. Not to mention (I dare not mention) management consultants. I'm beginning to think that a lot of it is misdirected, and can even be self-defeating.

How can I say that? Well, one reason is that I've been re-reading one of the most fascinating business books of the last 50 years -- The Innovator's Dilemma by Clayton Christensen. This is no careless work by a maverick writer. Christensen is one of the most respected business academics around, a Harvard professor loaded with honors and accolades. He tries to answer the question, why do great firms fail to keep market leadership when faced with disruptive technologies and other innovation? He dismisses the standard answers -- arrogance, myopia, poor values and procedures -- as being completely wrong. Instead, he says that the established firms he studied lost leadership because they were well run. "Precisely because these firms listened to their customers, invested aggressively in new technologies ... and because they carefully studied market trends and systematically allocated investment capital to innovations that promised the best returns," says Christensen, "they lost their positions of leadership."

What happened was that when a new technology or product came along, it was not as powerful or efficient as the established technology. Its products might be smaller, lighter, more convenient, easier to use, and cheaper, but performance was not as good. The established firms' customers therefore rejected the new product. There appeared to be no market for the new approach, and the only customers that could be found were in new applications at the bottom of the market. The customers and products were therefore unprofitable, the market for them was small, and their growth was uncertain at best. Applying rational financial criteria, investment in the simpler products could not be justified. The only firms willing to enter the new technology and provide the simpler products were new entrants -- mainly new ventures run by entrepreneurs - who had little to lose and hoped for the best.

Christensen's research showed that what then happened, time and again, was that the new products rapidly improved their performance, new customers appeared from nowhere to use the new products in new applications and market segments, the volume of the new products grew sharply (from a low base), the cost of the products tumbled, and their performance increased rapidly. The final stage was that the products became good enough to use in the old market (the one the established market leaders dominated), and the big customers who had rejected the new products started to buy them, because their performance was good enough and they were considerably cheaper. With greater volumes, costs continued to plummet and performance improved again. In many cases, the old market stopped growing, declined, or even disappeared. The erstwhile leaders were stranded, because by the time they could justify investment in the new approach, it was too late.

Christensen's research struck a chord with me because I am examining the broader question of what happens when simpler and cheaper products -- as well as better ones -- are pioneered, in preparation for a book on when, why, and how to simplify. I believe Christensen's conclusions apply to a much larger field than that of disruptive new technologies. For example, they also apply to Henry Ford's Model T car, to McDonald's, to IKEA, and to budget airlines -- none of which used truly new technologies -- and to most of the stunningly successful innovations of the past century.

Then I happened to be talking to a friend who is a venture capitalist. He told me that nobody in his industry uses conventional financial analysis, such as a DCF (discounted cash flow), to evaluate investments. The future market size and market share are just too uncertain. Modeling future profitability -- even crudely -- is a waste of time in venture capital. Usually the projections - especially of management teams -- prove to be far too optimistic and comically wrong. But when it matters -- when the firms turn out to make investors 10, 50, or 500 times their money -- the projections are always absurdly pessimistic. That is why the best investors never make them.

Then my VC friend and I got to discussing whether the capital asset pricing model (the academic theory behind DCF) is a useful representation of reality, or angels dancing on the end of a pin. I certainly remember working as a consultant and preparing incredibly detailed financial projections for major acquisitions that my clients wanted to make. In retrospect it seems to me that the spreadsheets were just a fig leaf of respectability for decisions that had already been taken. Those decisions might have been right or wrong - with the benefit of hindsight, I know that they were mainly wrong - but it bothers me that the financial modeling claimed to know something that was essentially unknowable.

At best, most of what we did, I now think, was an expensive waste of time. We were unwittingly misleading our clients. But we in turn were misled by following the best business academic theory. Isn't there something wrong with theory that lends itself to complicity in wrong decisions? Modeling future cash flows leads managers to think they have more control than they possibly can have, and to neglect the lessons that can be learnt about the wisdom or otherwise of a particular acquisition from other sources. Rules of thumb derived from long experience and thoughtful reflection, though not intellectually rigorous, can be much more useful than elaborate and internally consistent academic theory that cannot be reliably applied in practice.

So I'm coming round to a view that is subtly -- yet crucially -- different from that of Christensen. I don't think it is good enough to say that the firms that lost market leadership were well run, because they followed the best management practices. No. I draw two different conclusions, which cut the knees off from management science:

• If management science gives advice which leads well-run firms to fail, then that particular part of management science is no use. In fact, financial and market analyses systematically and predictably mislead managers and lead to horrendous mistakes. There are many reasons for this, but here are a few:

• It is impossible to predict new market sizes even approximately. A new market starts at size zero and that is the best time to get in - and create the market. Good innovators use principles that enable them to do this - but they know that essentially they are gambling. They do not use market or financial analysis, except sometimes to bamboozle uneducated investors.

• Paying attention to financial projections is the tail wagging the dog. If the product is good enough - if it offers at least one group of customers something that they prefer to existing products - it will succeed. If not, not. Financial analysis should always be subordinate to, and follow from, product design and analysis. As Henry Ford said, "the big thing is the product."

• Conventional market and financial analysis will always lead firms to be too conservative in innovation, and usually to miss the bus. Entrepreneurs have a huge advantage over established firms, because the former take more chances and go by instinct rather than by the rules of management science.

• It is better to observe what actually happens in the market place, and follow counterintuitive principles -- that are right more often than they are wrong, because they are based on observation -- than to engage in logical behavior consistent with best management principles.

• Which brings me on to my second disagreement with Christensen. Were the managers in his main examples, the disk drive industry, really performing well when they saw wave after wave of smaller disk drives taking over the previous market, and yet refused to participate? For sure, the first time it happened, when 14-inch disk drives were supplanted by 8-inch ones, the managers can be totally excused. But when 5.25-inch ones came along, might they not have learned to be so dismissive of the smaller, less powerful, and cheaper new products? And when the 3.5-inch, 2.5-inch, and 1.8-inch drives arrived, might the managers not have recognized an impending pattern? Good managers do not slavishly follow the rules of management science. They observe what works and what doesn't, regardless of what the textbooks say. They take a chance. They innovate. They break the mould.

But the real villain of the piece is probably not the managers. The real enemy may be management science itself, an oxymoron which everyone pays homage to, without understanding that there can be bad as well as good management science, and that science that leads to bad results has something wrong about it.

It may be that the most interesting and profitable parts of business cannot be reduced to a logical framework based on micro-economics and financial theory. If they could, all the important decisions would be taken by mathematicians and - heaven forbid - business strategists. Perhaps the most important decisions in business are creative and intuitive, not scientific or even logical.

All we can hope for are some principles that work, with greater or lesser reliability, in different kinds of circumstances. Some of these principles, such as the 80/20 rule, are incredibly powerful, can be empirically tested, and usually work. But the principles are isolated rocks of reason and insight in a sea of confusion and contradiction.

When you start thinking like this, it begins to raise disturbing questions. Is the whole body of management thinking in serious need of revision? Is most of what business schools teach really useful? Do highly intelligent and well-meaning consultants often lead their clients astray, or fail to warn them of very real dangers ahead, because there is no clever way of dressing up those dangers in the language of business academia?

It may not be true that the emperor has no clothes. But perhaps he is in serious need of a clothing makeover.

This will be my last blog this year. The next will appear on Tuesday 7th January 2014 and every Tuesday thereafter. May I wish you and your families a relaxing and enjoyable holiday break!