I guess you could have predicted this. With the world a mess -- call it disequilibrium, nonlinear perturbations, turbulence, possibly a phase change, perhaps a revolution, certainly a damn load of woe -- prediction as a respectable way to expend mental energy has suddenly become about as popular as Osama bin Laden futures. Now the truth is I've had serious doubts about the ability of anyone (including myself) to predict -- economists, analysts, especially pundits, most spectacularly anyone on television -- for some time now, certainly as long as I've realized the irrefutable fact that most stock pickers really stink, and that even the best have a lot of trouble sustaining a market-beating run. I would boast about this (well, I am) except that skepticism about prediction doesn't require genius, just a modest appreciation for history and a distrust of authority figures, like local weathermen and politicians. Living through the last decade has been one tutorial after another on the failure of prediction, in particularly, but not exclusively, the failure of markets to see around the corner: the dot-com bust, Sept. 11, the mortgage bubble, the financial crisis, the euro-zone mess, right on down to Hurricane Irene. In fact every decade teaches that lesson, though we are, as a species, very poor students in that regard.
That's a long preamble to the fact that the papers and blogosphere seem to be awash in denunciations of prediction today. The cover of this week's Bloomberg Businessweek is artfully apocalyptic in the run-up to the Sept. 11 anniversary, with a cover line for a story on reinsurance that declares, "Risk: A Decade of Disaster Has Made Predicting Impossible." Not a lot of nuance in that statement. In the Financial Times, the always-estimable John Kay, who was very early and sophisticated on such topics, hammers economists one more time Wednesday about why they're often wrong. Kay has come back from his holiday clearly re-energized to dismantle economic pretensions, as we've noted here and here. But in this column, he dwells on reflexivity generated by human systems when folks believe a prediction may be right, thus either leading to an efficient market or to predictions short-circuited by feedback loops. "The economic world, far more than the physical world, is influenced by our beliefs about it," writes Kay, who is nothing if not nuanced. "It is a mistake to ignore the efficient market; it is also a mistake to take it too seriously."
Andrew Sullivan, who does not usually paddle about in such waters (although skepticism about prediction does seem to be part of a certain kind of classical conservatism: If you can't see the future clearly, then be careful of advocating for change), gathers up a handful of posts from Robin Hanson's Overcoming Bias on personal prediction models and Erica Grieder at The Economist on journalistic prediction. Sullivan asks, "How can we make prediction more valuable?" He then links to a long and interesting summation of the issues from a blog called The Fifth Wave, which wrestles mostly with the difficulties of applying linear, Newtonian billiard-ball cause-and-effect concepts to nonlinear human events, that is to history. The Fifth Wave in turn links to two other attacks on prediction, Duncan Watts' book Everything is Obvious, and a book on punditry and its failings by Philip Tetlock, "Expert Political Judgment," that suggests that "great experts in world politics have been wrong often enough to put in doubt the whole concept of expertise." No knock on Tetlock, but that was pretty obvious.
Still, the Fifth Wave does set up the problem nicely:
In brief, we love to stretch common sense and Newtonian (or billiard-ball) causation beyond the breaking point. When we fail, we take it for granted it was because of insufficient information. This too is a failure of understanding. It's not that we lack enough information, it's that no amount of information can ever be enough. Human events unfold within complex systems governed by weird, nonlinear dynamics. Prediction by means of billiard-ball mechanics is impossible, in principle. Because each complex system develops in unique ways, events are also rarely susceptible to probabilistic analysis. Rightly considered, a question like "Who will win the 2012 presidential elections?" refers to a single token. There have been no previous 2012 presidential elections to average out with this one.
That said, when it comes to rescuing prediction from its seemingly irresistible quicksand, the Fifth Wave, channeling Watts, flounders a bit, arguing that analysts need a greater appreciation of narratives, local moralities and value judgments. Soon we are lost in the meta-debate over objectivity versus subjectivity and a vaguely philosophical exhortation "to gain sufficient familiarity with the emerging dynamics of the new system that he [Watts] can begin to assign a character to some of them." Well, that may well be correct, but it doesn't take us very far. It's like saying weathermen should focus on the storm bearing down on us, not the one that's in utero or longer trends like climate change.
More interesting here is the fact that, for the moment at least, the fallacies of prediction suddenly have gained currency, sort of like the Swiss franc or gold. What none of these folks really get, however, with the exception of Kay, is how deeply rooted both the urge to predict and how profound the limits to accurately predicting human behavior are. It's not just, as the Fifth Wave argues, that we need to abandon "the Western belief that we can stand, like God, above events." But prediction in some form or other is baked into nearly everything we do, certainly in business, finance and economics, but also in politics and punditry as well. We seem to need the faux certainty tossed up by the belief that there are dwelling among us -- in think tanks, in academia, in intelligence agencies, investment houses and corporations -- great minds that can see into the future or mechanisms, like the market, that through a magic we have never truly plumbed, possess a powerful prescience. What we also know about this phenomenon is how intimately it is tied to the Zeitgeist, to the great human market of beliefs and attitudes that shapes so much of our view of the future. It was no surprise that the market became an all-powerful prediction machine during the great bull market when optimism, geopolitically and technologically, waxed. And now that pessimism is flowing so strongly, we've suddenly realized we're children lost in fairy-tale woods without a working GPS. Anti-prediction attitudes are now rife. I predict this too will eventually change. Trust me.
Cross-posted from The Deal.