12/01/2014 04:23 pm ET Updated Jan 31, 2015

Why Change Is So Hard

When I was 18 years old, I spent my summer working in a steel mill. My shift was supposed to end at four o'clock, but on my first day, everyone began putting their tools away precisely at 3:15 p.m.

"Why are we quitting?" I asked.

"We need a half hour to clean up and fifteen minutes to shower," said the worker next to me. "We don't give away our time for free."

But it never took more than five minutes to put our things away and my colleagues would just sit there doing nothing, day after day. A decade later, that mill and most of the others in Pittsburgh shut down. High labor costs and low productivity made steelmaking a losing proposition in the Steel City.

I vowed to never again work for a company so resistant to change. But even though I have been lucky enough to spend most of my career at successful companies considered to be innovative, I have found that few employees jump at the chance to perform their jobs differently. Even at a leading technology firm, when we asked employees to adopt a new tool or process, most people came up with just as many excuses to hold onto the status quo as those idle steelworkers.

While virtually every company claims to embrace innovation these days, research over the last decade shows that most business change initiatives fail. Change doesn't take hold because of people--strong leadership or executive sponsorship is frequently cited as the most important factor for a successful initiative. The truth is most employees don't readily take on new behaviors without some help.

Countless business books have been written on driving change, but I recommend the three below to anyone who wants to understand why change is so hard for most of us. Organizations don't change unless their people decide to do things differently.

Start with Thinking, Fast and Slow, in which Nobel prize winner Daniel Kahneman describes two "systems" that run our lives. We'd like to think "System 2," our analytical mode of reasoning, is in charge. But in fact, our fast, intuitive, and largely unconscious "System 1" makes most of our decisions. Kahneman suggests it would be too hard to get through the day if we weren't running on autopilot most of the time. To change behavior, System 2 needs to pay attention. But System 2 is lazy and usually all too happy to accept the glib view of the world provided by System 1.

Once you understand those two modes of reasoning, you can move onto Switch: How to Change Things When Change is Hard. Authors Chip and Dan Heath describe their two systems as the Elephant and the Rider. Successful change requires directing the rational Rider and motivating the emotional Elephant. For example, you can direct your employees' rational side by finding bright spots in the organization and figuring out how to clone them. The Heaths also describe how an organization needs to shape the path for change by tweaking the environment and building habits.

If you have a little more time, you'll find some great examples of companies that helped employees behave more mindfully in The Power of Habit: Why We Do What We Do in Life and Business by Charles Duhigg. For example, Alcoa CEO Paul O'Neill created a keystone habit of safety by identifying a simple cue--an employee injury--and instituting an automatic routine of requiring a report within 24 hours. This not only made Alcoa one of the safest companies in the world, it changed other employee behaviors, resulting in greater innovation and profits.

More and more organizations are starting to realize that leadership just can't dictate change. Sixteen years of research by Prosci, a change management research firm, show that projects are six times more likely to meet objectives when organizations manage the people side of change effectively.

Innovation is certainly a great goal, but before we can go about transforming an organization, we need to do a better job of helping the vast majority of people who struggle when asked to do their jobs differently.