Websites Scramble As Hurricane Sandy Floods Data Centers

10/30/2012 07:49 pm ET | Updated Oct 31, 2012

As Hurricane Sandy made landfall Monday, the freak storm flooded data centers in New York City, taking down several major websites and services -- including The Huffington Post, Buzzfeed and Gawker -- that depended on them to run their businesses.

Several websites stored their data at a lower Manhattan data center run by Datagram, whose basement was inundated with water during the storm, flooding generators that were intended to keep the power on. In a statement, Datagram said no customers lost data and that it was working to restore its emergency power systems.

But the disruption clearly frustrated companies that relied on Datagram's services, and raised questions about why the company located its data center in a low-lying area.

“How dumb to locate datacenter in a flood zone. And how dumb to host Gawker servers there,” Gawker publisher Nick Denton tweeted on Tuesday.

George Reese, chief technology officer for enStratus, which helps companies manage their data, said companies often choose data centers based on cost.

"In an ideal world, people select data centers based on their tolerance of natural disasters," he said. "In reality, real estate and power costs drive data center locations, with disasters being something you prepare for."

"The critical thing is that you should never have all your data in one location or even in two locations susceptible to the same disaster," he added.

In July, an electrical storm that swept across the East Coast knocked out power at a Virginia data center run by Amazon Web Services, disrupting operations for numerous Internet companies that rely on Amazon's virtual data servers, including Instagram, Netflix and Pinterest.

Experts say failures at data centers can have widespread impact across the Internet, depending on how many customers the data center serves. Still, most companies, particularly smaller startups, don't back up their virtual infrastructure in multiple places, Reese said.

John Pavley, the chief technology officer for The Huffington Post, said its website began experiencing problems around 7 p.m. Monday when one of the site's New York City data centers was flooded by the storm, knocking parts of the site offline. His tech team scrambled to use the site's backup data center in Newark, but that center had its own problems -- three separate circuits that transfer data to and from the center had failed.

Several sites affected by the storm -- including The Huffington Post and Buzzfeed -- temporarily moved their operations to Tumblr blogs. Buzzfeed was able to restore its site by moving its data to Amazon Web Services, "with one developer working overnight despite a tree falling through his roof," Buzzfeed writer Matt Buchanan wrote in a post on Tuesday.

Datagram was not the only data center to experience flooding during the storm. Steadfast reported that servers at its data center in New York City were down on Monday due to an electrical failure.

As Consolidated Edison reported that parts of lower Manhattan faced up to four days without power, data center employees tried frantically to keep generators running. At one data center, employees reportedly hauled 55-gallon drums of diesel fuel up 17 flights of stairs to keep the generators running until electricity was restored.

For companies who rely on those servers, the stakes were high. Every hour their website was offline posed an increased risk that they will lose customers or readers, said Philip Jan Rothstein, a consultant who helps companies manage business risk in disasters.

"At some point, customers are going to say 'I’m going to get this elsewhere' or the equivalent, or 'Gee, I forgot, I’ve been using something else for so long, why would I go back?'"

Dino Grandoni and Bianca Bosker contributed to this report.

Did you lose power, Internet, cellular or other services as a result of Hurricane Sandy? Let us know what you lost, where, and for how long by tweeting to us @HuffPostTech or emailing us at Technology@huffingtonpost.com.

Also on HuffPost:

Hurricane Sandy

CONVERSATIONS