03/18/2010 05:12 am ET Updated Dec 06, 2017

Software Snakes On Planes 3 -- The Phone Interview

Homer Simpson Autothrottles

A phone conversation between literary agent Sam Fleishman and myself

Sam: "I just watched the video again. Let's recap your last two posts. In 2005, a Trojan Horse four-year-old autopilot computer bug sent this Boeing 777 skyrocketing. Then, though the pilot thought he'd turned the Homer Simpson autothrottles off, that computer stayed engaged, firewalled the throttles, forced the nose even higher and nearly caused a perfectly good plane to fall out of the sky. Two computers conspired to bugger this up and sent this plane on a roller-coaster phugoid?"


John: "Yes; MIT's Dr. Nancy Leveson explains that complex systems like this autopilot--autothrottle combo have caused accidents because of unplanned interactions across perfectly-functioning computers. She writes that cockpit computers have become so complex that they themselves introduce errors that can crash the system. And pilots can't figure out what's going wrong fast enough to stop what's going wrong."

Sam: "Scary; let's talk about these Homer Simpson autothrottles first. You showed me that though the pilot turned them off in two different places and a screen confirmed that they were off, they remained on. How can that be?"

John: "Craplex ergonomics; the pilot would've had to turn them off with a third switch."

Sam: "That's nuts; I expect my car's cruise control to quit when I tap the brake, not floor the gas pedal. Now you really are frightening me. How long have we known about these craplex ergonomic issues?"

John: "Since the 1996 FAA Report on Automation. I'll read from their page fifty . . ."

"You kept a fifteen-year-old report?"

John: "It's valid today. Anyway, that team wrote, 'Several crashes have come from multi-function knobs and controls with similar shape, feel, location and displays. These make it too easy for computers to hide problems from overstressed pilots until it's too late.'"

Sam: "'Until it's too late?' Ten years before this near-deadly phugoid and the FAA hadn't made Boeing fix these craplex 777 ergonomics?"

John: "The evidence speaks for itself; everybody dropped the ball."

"Golly, John, people nearly died. So what else does Doc Leveson say about this dysfunctional autopilot/autothrottle combo?"

"She explains that these were two reliable computers in an unsafe system and that software bugs strike randomly, without warning. She adds that software's inherent complexity makes it impossible to predict how it might crash."

"Frightening thought. So there are hidden bugs on planes flying right now that could strike any moment? Are passengers aware that they're rolling the dice?"

John: "Absolutely not; that's why I write this column."

Sam: "Here's what I don't get; the pilots must have seen what was going wrong on their flight and engine displays. Makes them look incompetent. After all, both CRTs were right in front of their eyes."

John: "You're not ready for that answer."

Sam: "O . . . kay; let's shift gears and talk about the software bug that triggered this phugoid. How could a 777 fly around for four years with a Trojan Horse bug? Wouldn't the pilots have gotten a warning?"

"Nope; the software healed itself, logged the bug in its data base, but kept the problem secret from pilots."

Sam: "How crazy is that? Why design a computer that keeps secrets from pilots?"

"Simple; manufacturers and airlines want planes to run on schedule, not parked at the gate with pilots whining to mechanics about spurious computer codes. Doc Leveson explains that self-healing software makes planes very reliable, but unsafe."

Sam: "But surely, John, the FAA and Congress acted on that '96 report."

John: "The trail of computercides says not; I'll write about them in this column. I find peace of mind, though, in Doc Leveson's insight that it's normal for plane-makers, governments and airlines to fiddle while Rome burns as we experience more computercides. Sad, but that's the harsh reality."

Sam: "Culminating in Air France 447 falling into the ocean from the Atlantic night sky; how awful. Not that I doubt you, John, but you're going to have to show me more proof that there's been no FAA action or Congressional oversight across the past fifteen years."

"The (cockpit automation) vulnerabilities we identified have the potential to lead to more accidents and serious incidents." -- 1996 FAA Report on Automation

Read all John Halliday's posts here:

(Phugoid image by Jeff Scott. YouTube video courtesy of Aardvark2zz)

Next: Craplex Cockpit Computers -- Unsafe At Any Speed