08/14/2014 12:46 pm ET Updated Oct 14, 2014

Irrational Markets Or Irrational Individuals? Designing 'Smart' Environments Improve Decision-Making and Make Markets More Efficient

How can markets be made efficient when people aren't? Apparent market failures are often attributed to individuals making apparent 'irrational' decisions. However, I argue that markets can be made more efficient when creating environments for individuals that aid their decision-making. When such 'choice architecture' is executed well, individuals can make decisions that more accurately reflect their intentions, ultimately making markets more efficient.

Every day news headlines are filled with yet another decision-making failure, spurred by cognitive biases individuals exhibit. We are too fat, too lazy, don't take our medication, and are emotionally driven in our decision-making. All of these decisions reflect actions that are not optimal for the individual. Many failures of decision-making stem from innate decision-making rules called 'heuristics', simple efficient rules used to form judgements. Although these mental shortcuts usually help us solve complex problems with ease, by processing heaps of information rapidly, decades of research originating in the work of Amos Tversky and Daniel Kahneman in the 1960s have identified areas where their application leads to systematic deviations from good judgement. To date, over 100 such cognitive biases have been identified, deviating individual's decision-making in ways that do not reflect best outcomes.

In recent years, markets have come under a lot of criticism as well. The financial crisis has often been taken as the pinnacle of market failure. For example, agents (professional fund managers) who manage investor's money, being detached from having real skin in the game, make trades that are too risky under the veil of ignorance, resulting in global banking scandals such as the Libor-fixing scandal in 2013. Why do these market failures occur? And how can we prevent such scandals to happen again, as indicated by the UK Treasury Minister Andrea Leadson?

A recent paper published in the journal PLOS One led by Prof Eric Johnson of Columbia Business School suggests a possible solution. Efficient markets assume that rational market participants process all available information, reaching optimal decisions based on all possible outcomes. However, our limited capacity restricts the information we can process and our cognitive biases distort the choices we make. In an analysis of health care exchange choices as part of the Patient Protection and Affordable Care Act, Johnson and colleagues found that individuals made decisions that were no better than chance, choosing the best option just 21 percent of the time. Hence, individuals intending to make a good health care plan suggestion fail to do so, being overwhelmed by the amount of information (in this scenario, individuals had 8 choices; the Massachusetts online exchange service presents 47 different plans, thus implicating even lower success rates for consumers). In their analysis, the study authors suggest that implies a market failure of about $10 billion.

Instead, Johnson and colleagues suggest to use the constraints posed by individual human beings, and design environments that may aid consumers make choices that more accurately reflect their intentions. This is the underlying idea behind 'choice architecture', which in opposition to the infamous 'Nudge' concept does not promote a certain option to people while retaining an element of choice (such as organ donation opt-outs), but instead focuses on striving to make the decision-making process easier. In the health care exchange scenario, this meant the limitation to just four health care plans (in order not to overload consumers), and the introduction of an online calculator, education about important terms in health care insurance (such as "What are premiums/annual deductibles?") and an introduction of a smart default, which pre-selects the cheapest choice based on an individual's input of their average data. And the results? Now, a staggering 80 percent chose the best option.

If people make decisions that more accurately reflect their preferences, can we prevent market failures? I believe that markets can function effectively only when consumers make decisions that accurately reflect their preferences (and not by choice, as in the health care example). Moreover, the additional price pressure and competition can help drive down costs for consumers, thus being reflective of efficient markets. Hence, it's not that markets are intrinsically inefficient, or that people make bad decisions per se. When presented with too much information, and without decision-making aids, individuals make choices that do not reflect their preferences and so are random.

Most environments where decisions are made are amenable to this approach. Take the energy-market in the UK as an example. The great amount of companies competing, and the number of different possible contracts each company offers implicates a great complexity inherent in this choice (which some comparison websites have tried to minimize). Moreover, the difficulty in accessing information about one's energy usage in a timely fashion distorts how decisions regarding energy-saving behavior are done. Unable to directly see effects, the lack of positive confirmation makes behavior change more difficult. Initiatives, such as British Gas Connected Homes aimed at collecting information and displaying it in the moment to consumers can provide a valuable first step at improving the choice architecture for consumers.

With smart choice architecture, we can not only help people make better decisions -- but we can also help create better, more efficient markets.

Jon M Jachimowicz