Many years ago, in my college yearbook, I listed my intended future occupation as "Bayesian soothsayer." This facetious choice stemmed from a desire to be both mysterious and noncommittal, while at the same time acknowledging an immediate plan to study statistics (the begetter of Bayesianism) in graduate school. In retrospect, I can see that my invented profession served its purposes well; but strangely enough, I now realize that it actually exists.
Since the term "Bayesian" is not well known outside the realm of statistics and its formal applications, but is central to the discussion that follows, a few words of explanation are in order. In its most general sense, the word describes anything associated with a famous theorem published posthumously by the Reverend Thomas Bayes (c. 1701-1761), an English mathematician. Bayes' theorem provides a convenient means of computing the unknown conditional probability of one particular event A given the observed occurrence of another event B, when the conditional probability of B given A (and all possible alternatives to A) is already known.
"Bayesian estimation" refers more narrowly to methods of statistical inference in which a decision maker combines both subjective prior estimates of an unknown parameter (such as the probability, p, that a particular coin comes up heads) with the outcomes of random experiments (such as the observed frequency with which the given coin comes up heads) to compute an updated posterior estimate of the unknown parameter. Although derived from Bayes' uncontroversial theorem, Bayesian estimation makes some statisticians uncomfortable because of its explicit use of subjective/judgmental assessments that can vary among decision makers. Such statisticians restrict themselves to methods of "frequentist estimation," which focus entirely on the outcomes of random experiments.
To see how soothsaying can insinuate its way into the Bayesian paradigm, imagine you are an ambitious ruler living in an ancient culture that frequently consults oracles (such as the Pythia of Delphi). Suppose you are contemplating the siege of a neighboring city-state, and that you estimate the probability of success to be very high -- say, about 90 percent -- based upon a subjective assessment of the relative strengths of the opposing armies. (For example, you may believe this probability is equally likely to be any number between 85 percent and 95 percent.) Unfortunately, however, your ministers and generals are not so sanguine, and have informed you that, in all of history, the enemy city has been attacked ten times, but defeated only once. Thus, from your advisors' frequentist perspective, a 10 percent chance of success is more reasonable.
As a self-confident leader, you most likely will give substantially more weight to your prior probability of success than to that of your advisors in forming a posterior estimate. For instance, an 80-20 weighting scheme would result in a posterior estimate of 74 percent (= 90 × 0.80 + 10 × 0.20). However, there remains one problem: how to rationalize this choice without appearing to have ignored your advisors.
This is where divination methods come in. By providing a third assessment that is likely ambiguous and/or cryptic (such as the Delphic Oracle's prediction to Croesus: "If you [attack the Persians], you will destroy a mighty empire."), a soothsayer affords political cover for your personal judgment. Moreover, if you happen to be skillful in assessing prior probabilities, then the soothsayer's foresight, which you have interpreted to support your prior estimate, will appear quite accurate in hindsight.
But what does this discussion of city-states and oracles have to do with contemporary decision making? Somewhat disquietingly, there is in fact a modern equivalent of soothsaying in the social sciences: the selection of data sets and/or econometric techniques designed to reinforce one's prior estimate. Naturally, we could address this issue in the context of highly controversial socio-economic problems, such as the impact of government fiscal policy, the war on drugs, health-care reform, etc. However, to remain untainted by the nastiness of contemporary politics, let us consider a passion-free sports example: the probability that the Boston Red Sox will win the 2012 American League pennant race.
Suppose you are very optimistic about the Red Sox' prospects, and that your prior estimate of the desired probability, based upon a subjective assessment of the relative strengths of all major league baseball teams, is about 90 percent. Subsequently, upon analyzing the most recent seventeen years' data (since baseball's rules were adjusted in 1995 to permit post-season wild card teams), you find that the Red Sox have won the pennant only twice in that period, for a success rate of about 12 percent. Finally, employing the same 80-20 weighting scheme suggested above, you compute a posterior estimate of 74.4 percent (= 90 × 0.80 + 12 × 0.20). How can you convince others that such optimism is warranted?
Like divination methods, the selective selection of data and/or forecasting methods affords a third assessment to justify your personal judgment. For example, you may build a model that shows the Red Sox do particularly well in post-season play whenever they are a wild card team, and that current standings favor such an outcome. Alternatively, you may identify one or more seemingly relevant exogenous variables that happen to have peaked in 2004 and 2007 (the years the Red Sox previously won the pennant) as well as in the present year.
What is particularly ironic about all of this is that there is nothing intrinsically wrong with the Bayesian (i.e., subjective/judgmental) aspect of Bayesian soothsaying. (In fact, I would be happy to be called a Bayesian, but not a Bayesian soothsayer, ... despite what I may have written in my incautious youth.) Rather, it is the unacknowledged or secretive use of subjective information that is troublesome, whether perpetrated by Bayesians or frequentists.