Robots to the Rescue: Five Lessons From Hurricane Katrina (and 22 Subsequent Emergency Robot Deployments)

09/15/2015 04:22 pm ET Updated Dec 06, 2017

2015-08-31-1441057528-3982580-robin_murphy_h.jpg

Robin Murphy is the master of disaster robots.

After 9/11, she deployed small mobile robots to investigate the rubble.

In the wake of Katrina, she sent small, unmanned aerial vehicles to explore buildings on the Gulf Coast - the first time UAVs had been used for emergency structural inspections.

2015-08-31-1441057690-6644498-Katrina16.jpg

During Fukushima, she was part of a team that flew UAVs to determine radiation levels and inspect damage to the reactors.

And during the recent Ebola pandemic, she explored whether robots could help protect health workers from contagion in field hospitals in West Africa and in the U.S.

2015-08-31-1441058842-5141815-Fukushima3.jpg

As director of Center for Robot-Assisted Search and Rescue (CRASAR) at Texas A&M University, Murphy has played an active role in these and more than 20 other disasters, but she's first and foremost a researcher.

Using her experiences in these hot zones as a starting point, she returns to Disaster City, the state-of-the-art disaster reconstruction site at Texas A&M, and probes what works and what doesn't in order to determine how to most effectively use robots to hasten rescue and recovery efforts.

2015-08-31-1441057549-5662824-disaster_city.jpg

Based on seminal disaster studies, she estimates that speeding up an emergency response by one day through the use of autonomous robots and unmanned machines can reduce the recovery time by 1,000 days or almost three years.

What follows are five lessons she learned from her experiences and research in disaster robotics:

1. The number of experts it takes to operate a drone. (Three!)

Before Katrina, no one knew for sure how many people were required to operate an unmanned aerial system in a disaster situation.

"The Army thought you just needed one person to fly, but it turns out that you need three people to do it effectively," she said. "When we started doing the workload analysis -- which we were the first to do -- we realized one person just can't do it."

Safely flying a drone in close proximity to a damaged building while taking photographs was an impossible task for just one person, she found.

Her insights didn't win her any popularity points, as it made operations more complex and costly to run, but her follow-up research proved her right.

"Our protocol of having one to fly, one to look and one to serve as the safety officer has now become standard practice throughout the world," she said.

And having an accurate sense of the resources needed for operations is critical to getting the useful information that can save lives and speed recovery efforts.

2. It's the data, not the drones.

When UAVs were deployed for the first time after Katrina, they were still a relatively new technology. Problems with power and stability were among the concerns of Murphy and other scientists.

"Before Katrina, the hard problem was keeping the platform in the air long enough to be useful," she said. "Now the problems have shifted from the aeronautical science to the data science."

Our connected world means that experts anywhere can start looking at imagery collected by drones and other sources over the Internet and aid in recovery efforts as soon as the data becomes available. That kind of cooperative effort can shave weeks or months off of the process of inspecting buildings or identifying areas of concern.

But what kinds of remotely collected information are helpful? And how can that data be served up so it's useful to experts? Those are still questions fundamental to Murphy's research.

"Katrina showed me that the hardest problem is getting the right data to the right person at the right time," she said. "That was true then and is still true."

Better ways are needed to collect, transmit, collate and visualize information. As she put it in her recent TED Talk: "It's not the robots, it's the data."

3. Understanding what experts need to see and when they need to see it.

In the months after Katrina -- supported by a grant from the National Science Foundation -- Murphy deployed drones to transmit images of damaged multi-story buildings on the Gulf Coast to structural experts at FEMA, leading engineering firms and universities.

The work shortened the time required to get the buildings examined, but structural engineers found it harder to use the drone imagery than expected -- showing Murphy that the problems of drone reconnaissance were more complicated than many believed.

"When you look at stuff on the ground, things pop up at you," Murphy said. "With UAV photos, that wasn't happening."

Recently, Murphy has experimented with letting experts direct drones in real time by circling or spotlighting locations on the live video feed. She found this "visual common ground" between the experts and the operator helped significantly.

"Recognizing that the collaboration between humans and machines is only possible when we're all looking at the same feed from the UAV or the ground robot -- that's been a big breakthrough," she said.

4. The future of robots is underwater.

Robo-snakes that can comb through rubble and infrastructure-inspecting robotic hummingbirds will play important roles in future emergency response situations.

But autonomous marine vehicles have thus far presented a missed opportunity for roboticists, Murphy believes. She says they represent the next big thing in disaster robotics.

"Eighty percent of the population lives by water," she said. "All your bridges, your levees, your critical infrastructure are underwater and we're relying on manual divers, of which there are very few, to go inspect them."

For these reasons, she and her team have experimented with marine robots, called SARbots -- square robots with sensing capabilities like dolphins. They go underwater, use sonar and can be controlled remotely to rapidly investigate water-based risks.

In 2011, after the Tōhoku earthquake and tsunami, Murphy worked with the Japanese Coast Guard to use the underwater robots to inspect and clear ports and look for victims.

"Four hundred miles of coastland totally devastated, twice the amount of coastland devastated by Hurricane Katrina in the United States," Murphy said. "You're talking about your bridges, your pipelines, your ports -- wiped out. And if you don't have a port, you don't have a way to get in enough relief supplies to support a population."

In the future, autonomous marine vehicles may prove as valuable as other types of robots now commonly in use in disasters.

5. Smarter algorithms should tame the flood of data.

In the days following the 2015 floods in Texas, Murphy was part of a team that deployed UAVs to inspect the damage.

She found that one 20-minute drone flight generated upwards of 800 photographs, each of which took at least one minute to inspect.

When you consider the possibility of flying dozens of drones in an effort to save lives, such a system just isn't scalable.

"It takes a huge amounts of time just to go through the data," she said. "So we need to make improvements in the data science and informatics."

It just so happened that Murphy was running a research program for undergraduate students when the floods occurred. She asked her students to develop applications that could improve the processes of operating UAVs, interpreting their footage and even differentiating areas that need human insight versus those that don't.

The students rose to the challenge, coming up with apps that used computer vision and machine learning to improve the recovery process.

2015-08-31-1441057723-6351597-fom_1.png

"They took the data as it was happening and created programs that looked for any signs of debris that might suggest a person might have been swept along nearby," Murphy explained.

Other students from the program created improved flight path visualizations to help responders understand where drones had already flown to avoid duplication.

The students earned top marks not only in her class, but also in a competition for recovery innovations.

In addition to student projects, researchers at the University of Maryland used "deep learning" algorithms and anomaly detection techniques to design a system to that allowed UAVs to find piles of debris big enough to hide a body. The algorithms offered significant advances over technology considered state-of-the-art.

"The responders said 'anything that helps us zero in on people is better than nothing,'" she said. "If the computer can process 70 percent of the images, that 70 percent less work for them."

It's far from a solved problem, but it shows how valuable machine learning and computer vision will be for search and rescue missions in the future.

--

Robots haven't saved any lives in disasters yet (although a UAV did help identify a Canadian man after he'd wandered away from his disabled car and gotten lost in the snowy woods), but they're sure to do so in the future.

"The next time you hear about a disaster, look for the robots," Murphy said. "They may be underground, they may be underwater, they may be in the sky, but they should be there. Look for the robots, because robots are coming to the rescue."

--

Images/videos shown above (top to bottom):

Researcher Robin Murphy studies robot-human interactions, including during disaster scenarios, using helper robots like this "Disaster Buddy." Credit: Texas A&M University

An unmanned aerial vehicle operated by Safety Security Rescue Research Center (SSRRC) team members captured this image of devastation in Pearlington, Miss., during a search for survivors of Hurricane Katrina. SSRRC is one of the National Science Foundation's Industry-University Cooperative Research Centers. Credit: Safety Security Rescue Research Center/National Science Foundation

Led by a trainer in a protective suit, the General Dynamics Land Systems MUTT was demonstrated as one way to use a robot to transport a patient. Credit: Texas A&M

The sign that greets you at the entrance to Disaster City in College Station, Texas. Disaster City is where the Texas Engineering Extension Service (TEEX) is located and is one of best emergency response training facilities in the world. Credit: Texas Engineering Extension Service (TEEX)

Video footage of the Isensys UAV used during Hurricane Katrina. Credit: Robin Murphy

Robot's eye view of a UAS flight from the mudslides along State Route 530 is a Washington state on April 23, 2014. The mission helped hydrologists determine the state of the river and the potential for flooding responders working downslope. Credit: CRASAR

Footage from a hazmat exercise using AirRobot 100 showing a new app, Skywriter Sketch, used to help pilots and team-members speed reconnaisance. Credit: Robin Murphy

Post tsunami relief aid in the form of searching through debris in the water. A house was destroyed and floating in a small cove. The SeaBotix SARbot swims inside to investigate. Credit: Teledyne SeaBotix

Student Abygail McMillian created a program that visually displays where and when UAV data or any geotagged data was taken. Credit: CRASAR

A collection of video footage from robot deployments between 2001-2015 led by CRASAR a Texas A&M Engineering Experiment Station Center whose mission is to improve disaster preparedness, prevention, response, and recovery through the development and adoption of robots and related technologies. Credit: CRASAR