THE BLOG

Let's Not Welcome Computers as Our New Overlords Just Yet

09/06/2011 01:25 pm ET | Updated Nov 06, 2011

There was a brilliant article recently on the BBC website posing the theory that algorithms are taking over the world. It is true that algorithms are now being used for everything from finding friends on Facebook to making trading decisions based on Twitter sentiment. High frequency trading, the recent media bad boy, is an algorithm-fueled method of buying and selling stocks - among other things. But algorithms are not quite the world-dominating powers that some people fear - yet.

I love algorithms. They are the computerized version of the human brain's thought process, taking a set of rules and using them to make and execute a decision. They can help with social activities like helping you to decide which films you might enjoy when you sign into Netflix. They can set you up with people that you may enjoy dating, or even marrying, like on Match.com.

They also have some incredibly practical uses, such as automating fairly standard business processes. An algorithm would be behind checking to see if one of your trucks had reached a certain depot on time. Or an algorithm can execute your trading decisions and make sure that you buy or sell at the best price. An algorithm can also sniff out and email you relevant jobs that are being advertised online.

The fear today is that algorithms could be overused, taking the creativity away from the decision-making process. They could turn us humans into mush-minded creatures who can't be bothered to make our own choices. If algorithms are as powerful as some say, couldn't they one day replace humans with so-called artificial intelligence?

Ten years ago, British scientist Professor Stephen Hawking warned that if humans were not genetically enhanced they would not be able to compete with artificial intelligence. He said that the increasing sophistication of computer technology is likely to outstrip human intelligence in the future. He told the German magazine Focus on September 3, 2001: "In contrast with our intellect, computers double their performance every 18 months. So the danger is real that they could develop intelligence and take over the world."

With all due respect to Professor Hawking, who was one of my heroes at Cambridge University, I don't think so. Yes, human decisions may be slower than algorithms. But human intuition is the secret sauce in decision management - including validating that certain algorithmic decisions make sense.

One of my followers on Twitter, with whom I regularly exchange ideas, recently tweeted "Ten years ago I met my amazing wife on Match.com. She was rated a mere 50% match". So we'll give the algorithm an A-. It made an excellent suggestion but couldn't see further than a few surface criteria to assess compatibility. The remaining 50% of the work to determine compatibility had to be done the old-fashioned way. Would the Match.com algorithm do better today, I wonder?

What if the new algorithm rated my friend and another woman as a 90% match, would that mean he would simply trust the algorithm and go straight for a proposal of marriage? That would certainly take the responsibility of choosing his wife off of his shoulders. But it is hard to blame an algorithm when you make a mistake.

If an algorithm that is monitoring Twitter sentiment tells you that you should sell your Apple shares because the probability of Steve Jobs retiring is 80% higher today than it was yesterday, would you sell? Even if it were correct you could lose money. If you had sold your shares on August 24th, the day after he announced his retirement, you might have received $350 per share. Had you made a more human decision to wait and see what happened, you might have sold them a few days later at $390. You would have lost $40 by listening to the Twitter sentiment algo.

The real danger is that technology can make us lazy. Blindly following an algorithm is like driving into a lake because your GPS told you to (true story!). If you do not use intelligence - human or artificial - to set the correct parameters of an algorithm, it can take you down the wrong path.

In financial services we have seen this already. The May 6th flash crash occurred because a mutual fund (Waddell & Reed in Kansas) entered a large ($4.1billion) sell order in E-mini S&P 500 futures contracts on the CME. On a bearish day, it sparked an algorithmic and human selling spree. The fund was using an automated algorithm through its broker to slice up the large order into smaller batches, in order to minimize the impact on the market. However, it is believed the parameters of the algorithm were badly defined, making the batches too large and putting almost the whole order into play at once. Had the proper risk controls and monitoring technology been in place to catch the mistakes that the algorithm was making the flash crash might never have happened.

The massive usage of trading algorithms is a major factor behind the huge volatility we've seen recently in the market. The "sheep mentality" of the human traders has been accelerated and accentuated by the sheep behavior of computer algorithms. The flash crash was an extreme example of this.

And high-speed algorithmic markets that lack adequate supervision (real-time monitoring and surveillance) risk even scarier things than flash crashes. Without trying to cause panic, in the most extreme circumstances abusive practices could be considered algorithmic terrorism. The concern is that a well-funded terrorist organization might use such tactics to manipulate or cripple a market. So much of our economy is underpinned by electronic trading that protecting the market is more important than guarding Fort Knox.

We must ensure algorithmic monitoring systems are in place that identify the high risk decisions and dangerous circumstances, and bring humans into the loop. Rather than Professor Hawking's augmented humans, the key challenge is systems to augment algorithms with human intuition.

Risk is everywhere, not just in financial services. Marrying the wrong person, or taking a terrible job, or renting a lousy movie because an algo told you to is irresponsible, but perhaps not dangerous. If, however, you hand over your decision-making processes to algorithms without employing any sensible controls and monitoring, this is where danger lurks.