09/08/2014 01:07 pm ET Updated Dec 06, 2017

What Baseball Teaches Us About Big Data

Let's face it, Major League Baseball -- and those who follow it -- like tradition.

Baseball is all about poetry in motion, the cut of the grass, the timelessness of the game, the historic heroes and the manager's gut. Sometimes, a manager just has to go with his gut. He knows a pitcher in the bullpen owns the pinch-hitter coming to bat. He knows the wheel play is on and a bunt isn't the wisest course just now.

But baseball, like all professional sports, is about something else, too. Winning. And it turns out one way to increase the chances of doing that is to turn away from the gut and toward the data. Oakland Athletics general manager Billy Beane and author Michael Lewis did more to promote that thinking than anyone, as outlined in Moneyball: The Art of Winning an Unfair Game. (The view, of course, applies to other businesses, as well.)


I was reminded of this over the weekend while listening to baseball guru and KNBR-AM radio host Marty Lurie and his interview of Arizona Diamondbacks' coach Mark Weidemaier, who was talking about a baseball hot-button: The practice of employing an infield shift to better defend against left-handed pull hitters. The move, in which three defenders (or four in extreme cases), take positions between first and second base, appears to be working.

And it makes sense: Left-handed hitters naturally hit the ball to the right side of the infield. But Weidemaier said he's not going with his gut when he sets the Diamondbacks up in a shift.

"I'm in charge of putting together the defensive alignments, our optimal positions, where we're going to play to start a ballgame against each hitter," Weidemaier, who's been in the game for three decades, told Lurie. "We look at all the spray charts. We look at all the data."

He added that looking at charts that show where on the field hitters hit the ball is nothing new. Coaches and managers have been jotting that down for decades. But something has changed since the days of dugout clipboards.

"Now you have the computer, which gives you these charts at a click of a finger for 4,000 at bats," he said. "When you see the shading, when you see the charts, it's wise to take advantage of the data."

It is wise, but not necessarily universally accepted by fans and pundits. The idea that stats can outsmart baseball brains is the source of a heated and ongoing debate. How can a machine outperform years of playing and coaching ball?

(Turns out that some wonder whether the shift should be outlawed, while others believe the game will right itself as it always has.)

Sound familiar? Whether the field is education, climatology, politics, investing, journalism, retail or baseball, there are people involved, people who develop expertise and experience. People, who at times, trust their guts.

Chicago White Sox announcer Ken "Hawk" Harrelson, for instance, isn't real big on data as he explains in this video debate with the MLB Network's Brian Kenny.

And maybe these sorts of all-or-nothing debates unintentionally make the most important point: The answer is not to give over all decision-making to the machine. Nor is it to completely disregard the data and go with the gut every time. The secret, as John Kay points out in the Financial Times, is to combine the best of both types of intelligence to give an organization the maximum chance to succeed.

Lurie, who's as good as they get when it comes to baseball knowledge, touched on the point with Weidemaier, when he pointed out that in order for the shift to work, the team's pitcher needs to put the ball in the right place to ensure the batter indeed hits it to the right side.

"To do it in harmony," with the pitcher, "is the best way to go about it, obviously," Weidemaier said. "We try to put our guys in the optimal position to begin with and then work off that as for how our pitchers attack hitters during the game. And I keep charts during the game myself. As the year goes on, I build off of that."

And as he builds profiles of hitters, he shares them with the game's pitcher and catcher (they meet before every game) and the team's defensive players (they meet before every series) to ensure that the human element is very much a part of the equation.

It sounds like Weidemaier has found the sweet spot: Let the machine do the grunt work of tracking thousands of at bats, while he puts his considerable experience to work helping pitchers and defenders understand what it all means for them.

At that point, the debate is no longer about which is better, the human or the machine. At that point the question is how can one best improve the insights provided by the other.

Mitt photo by Andrei Niemimäki published under Creative Commons license.

This post originally appeared on the BloomReach blog. Mike Cassidy is BloomReach's storyteller. Contact him at and follow him on Twitter at @mikecassidy.