From Research to Reality

Federal IT R&D Technologies Play Key Roles in Disaster Response

In the hours immediately following the September 11 terrorist attacks, four unusual rescue teams sped toward New York City in cars, vans, and planes from as far away as California and Florida. Each team included computer scientists and a collection of miniature robotic vehicles, many of them first-generation prototypes developed with funding from the Defense Advanced Research Projects Agency(DARPA), the National Science Foundation (NSF), and the Office of Naval Research (ONR). At Ground Zero, the teams' work was coordinated by the Federal Emergency Management Agency (FEMA).

Over the next two weeks, the teams’ shoebox- to suitcase-sized, track-wheeled vehicles penetrated up to 45 feet inside the mountainous, fiery devastation of the World Trade Center buildings through openings too small or too hazardous for human rescuers to investigate. The density of the collapsed rubble rendered the larger robots useless, but the smallest ones could be inserted into spaces less than a foot wide. The vehicles sent back video images of their surroundings and many carried microphones, transmitters, and infrared sensors to detect any signs of life and enable communication to and from the surface. They were able to find pockets of space where survivors might be and to identify structural conditions that posed dangers to those trying to remove debris. But like the rescuers working frantically at the surface of the gigantic smoking piles, the robots found no life, only remains.

The World Trade Center effort marked the first known use of robots in an urban search and rescue operation. Other technologies pioneered through Federal information technology (IT) research and development (R&D) investments provided advanced scientific maps and hazard assessments at Ground Zero and the Pentagon that were critically important to emergency-response activities after 9/11.

The advanced technologies - robotics, global positioning satellite (GPS) technologies, optical technologies for remote sensing and environmental monitoring, high-speed networking, computational chemical analysis techniques, and mapping, modeling, and simulation software- used in three separate disaster response activities described here originated in Federally funded IT R&D. In each effort, diverse information technologies working together provided the human responders with essential, timely, and otherwise unavailable findings to guide their decisions and actions. These IT capabilities made it possible to rapidly gather vast quantities of real-time data through remote sensing devices, to process and analyze the data, and to represent the scientific results g raphically - transforming raw information into timely tools for human understanding and informed action. Federal IT research generates a continuing cycle of such advances from bench research to prototype demonstration to widescale deployment. The constant developmental effort supports critical Federal missions,seeds privatesector enterprise, and provides the lift for what John H. Marburger III, director of the White House Office of Science and Technology Policy (OSTP), calls the U.S. "trajectory of world leadership in science."

The robotics teams were called into action by a DARPA program manager specializing in robotic systems who had developed a strong interest in the potential applications of robots in urban search and rescue operations. DARPA's ongoing R&D in small, portable, unmanned systems aims to increase the agility, mobility, and autonomy of these systems for the tactical battlefield while decreasing their size - characteristics that also are relevant in disasters.

On September 11, DARPA Director Anthony J.Tether quickly approved the idea of sending an agency team with some DARPA prototype robots to the World Trade Center. Other robotics specialists also were called on, including Navy researchers in San Diego, personnel from the Massachusetts technology firms Foster-Miller and iRobot, and

Dr. Robin Murphy, a leading robotics researcher at the University of South Florida whose work has been supported by DARPA, the National Aeronautics and Space Administration (NASA), ONR, and NSF. In addition to conducting her own research, Murphy introduces undergraduate and graduate students to the theoretical and technical issues of robotics engineering, in part with funding from NSF's Research Experiences for Undergraduates program. On 9/11 the professor of computer science and cognitive neuroscience loaded her van with graduate students and robot prototypes and drove straight through to the teams' rendezvous site, arriving the morning of September 12. As things turned out, an overriding issue for all the robotics specialists at Ground Zero was platform size: only 7 of the 17 vehicles brought to the site were small enough to be used amid the dense rubble.

Since 9/11 the value of robotic devices for search and rescue work and remote reconnaissance in hazardous conditions is increasingly recognized within the Department of Defense and in the emergency response community. For example, Murphy now works as a volunteer to train local police, fire, and other first responders in urban search and rescue techniques, including uses of robotic devices. And she and other robotics researchers examining the technical lessons of the World Trade Center experience are making use of a portable test course established in 2000 by the National Institute of Standards and Technology (NIST) to evaluate robots' functioning in disaster environments. NIST, DARPA, and robotics research groups use the reference-testing facility to develop objective performance metrics; several of the Ground Zero robots had been run through the course prior to the disaster.

Although current technology shows significant progress over earlier generations, the systems employed in New York were still research platforms and exhibited serious shortfalls. Murphy noted at a recent NSF briefing that, although the prototype vehicles successfully contributed to the Ground Zero crisis response, their frustrated operators took away a long list of needed improvements requiring a continued advanced research effort. The research issues include durability (e very robot used was damaged); speed, power, and maneuverability (robots flipped over and could not right themselves, or got stuck, or could not surmount obstacles); computational component issues (the smallest robots had no onboard processing, so they had to be wireline-tethered to surface-level controls, limiting mobility); camera occlusion and poor image quality; inadequate sensing capabilities (disaster-response robots need to be able to relay fine-grained and 3-D information about topology, materials, and other environmental characteristics, as well as about their own situation and physical state); and serious software problems (interfaces were difficult to use and software was not interoperable across robots).

As in tactical battlefield situations, the promise of robotics in disaster response is to undertake critical tasks, such as real-time information gathering and transmission, under conditions that are life-threatening for humans. Murphy noted that in the 1985 Mexico City earthquake, 65 of the 135 rescuers who died lost their lives inside damaged buildings. "It's not about the robots," Murphy said. "They are just another tool for emergency rescuers to use. Urban search and rescue is all about information - and the information technology it takes to get findings that rescuers urgently need into their hands right away."

Shortly after the terrorist attacks, the geosensing systems unit in the University of Florida (UF) Department of Civil and Coastal Engineering received a phone call from the Joint Precision Strike Demonstration Group of the Department of Defense (DoD) asking for its laser-mapping expertise in a collaborative effort with Federal agencies to develop advanced scientific maps and damage and hazard assessments at the World Trade Center and the Pentagon. The UF researchers, whose Federally supported work applies scanning laser technologies to topographical and structural analysis, joined personnel and equipment from the National Oceanographic and Atmospheric Administration (NOAA), NOAA's National Geodetic Survey, and Optech, Inc., a Canadian manufacturer of laser devices, in the data-collection and analysis activity. Optech provided airborne and groundbased laser scanning instruments, and NOAA contributed a plane and pilot to collect the airborne observations. NSF added some "quick response" funding to enable the UF team, which is developing a novel 3-D structural mapping technique, to make additional laser observations and conduct a study of the results.

At the same time, Robert O. Green, a scientist at NASA’s Jet Propulsion Laboratory ( JPL), was on the phone with research colleagues at the Environmental Protection Agency (EPA) and the U.S. Geological Survey (USGS) discussing what contribution the three agencies might make to help emergency officials understand the environmental conditions at Ground Zero. They agreed that NASA, USGS, and EPA technologies and analytical capabilities could provide assessments of friable asbestos levels at the site and obtained a goahead from officials at their agencies to do the work. As the activity developed, however, the scientists realized there were other significant issues - such as the locations and temperatures of fires burning at Ground Zero, and the types and dispersion of airborne particulates - about which emergency workers needed strategic scientific information for decision making. FEMA and OSTP officials, recognizing the utility of the initial findings, asked the Federal scientists to continue collecting data over the following week.

Information technologies applied in these two collaborative Federal efforts included NASA's advanced remote sensing capabilities, DoD's global positioning satellite (GPS) technolog y, USGS's worldleading topographical and chemical analysis techniques, and computer modeling and visualization technologies developed by DARPA, the Department of Energy (DOE), NASA, NOAA, and NSF. These capabilities, along with advances in networking and computational power driven by IT R&D, enabled scientists involved in each of the mapping and hazard-assessment projects to turn very large and complex streams of sensor data rapidly into site maps and analyses.

NASA’s Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) was flown by its JPL team over the Trade Center area on September 16, 18, 22, and 23. AVIRIS is a unique hyperspectral remote sensing instrument that delivers calibrated images of rising spectralradiance in 224 contiguous bands with wavelengths from 400 to 2,500 nanometers. (The various bands make possible high-accuracy measurements across a wide variety of target surfaces, ranges, and atmospheric conditions.) After the initial flyovers, the Federal team was able on September 18 to provide emergency workers with the first information identifying three dozen thermal "hot spots" at Ground Zero, ranging from 800 degrees to more than 1,300 degrees Fahrenheit. By focusing on the targeted hot spots, firefighters were able to reduce temperatures to near-normal by September 23, as verified by the hyperspectral data collected in NASA's flyover on that day.

Computational tools enable data analyses

Digital spectral data collected in the NASA flights were delivered by FEMA courier to JPL in California, where Robert Green and colleagues worked through the night to process the findings via high-performance computing. Then they were networked to the USGS Imaging Spectroscopy Lab in Denver, where scientist Roger Clark and colleagues performed computational analyses including surface reflectance calibration and spectral mapping. Physical samples collected at the site by USGS personnel also were analyzed for mineralogical and chemical properties using such technologies as reflectance spectroscopy, scanning electron microscopy, and X-ray diffraction.

On September 23, the researchers provided World Trade Center emergency workers with complete maps and reports detailing the substances in the air and on the ground. The results were consistent with the EPA ground team's assessment that only trace levels of asbestos were present in the air. But the spectral and sample analyses provided additional significant details, pinpointing the precise composition and dispersion of particulates. Other particulates included glass fibers, gypsum, concrete, paper, other substances commonly used in construction, and some heavy metals such as chromium and molybdenum. The analyses found that there were higher concentrations of asbestos on beam coatings but that the Ground Zero asbestos was composed only of chrysotile asbestos, which studies have found to be less carcinogenic than other forms. The interagency team agreed, however, with New York Public Health recommendations that cleanup work should be done with appropriate respiratory protection and dust control measures.

ALSM technologies provide accurate views of damage to buildings

Meanwhile, the University of Florida group was working with a combination of imaging and computing capabilities it has developed to generate a new type of 3-D map that can be particularly useful in assessing structural integrity and vulnerabilities.The technique starts with airborne laser swath mapping (ALSM), also called light detection and ranging (LIDAR), a line-of-sight technology first developed by NASA and USGS in the 1980s for wide-area topographic mapping. In ALSM, a pattern of narrow pulses or beams of light (of much finer resolution than the broad radio waves of radar) is directed to the surface being mapped. A digital optical receiver in the plane records, counts, and times the returning light pulses. Since light travels at 30 centimeters per nanosecond, high-speed computation and visualization software can convert these data points into high-resolution maps of topographical features.

At the Pentagon, the NOAA aircraft made 30 ALSM scans totaling more than 31 million data points, and an Optech crew conducted 13 ground-level scans. The company processed the data rapidly, providing Defense officials within days with topographical and eyelevel images of Pentagon structural damage.

Turning laser dots into precision 3-D digital elevation maps

Back at the University of Florida's ALSM processing lab, the hundreds of millions of data points gathered at the World Trade Center underwent intensive computational processing and filtering, using algorithms developed by the researchers to eliminate irrelevant topographical "noise" such as cars, airborne particulates, and vegetation, and to generate georeferenced data grids. The output images look like black and white photographs, but they are in fact vast clouds of data points, each with its own spatial coordinates on three axes, accurate to within a few centimeters.With these 3-D digital elevation maps, a viewer can look at a structure from above and then zoom in to see the same view from other angles. The map user can search for cracks and fissures, estimate the volume (and thus the mass) of debris fields or holes, and superimpose a geometric mesh over a structural element to compute its degree of deformation or deflection (thus stress) and the size and shape of surface damage.

As part of the NSF "quick response" funding, the University of Florida researchers will recommend ways that their innovative mapping system can be used in future disaster prevention and mitigation efforts - such as mapping potential earthquake and flood zones and evacuation routes - as well as in emergency response situations.

The coordination of Federal IT research investments across many agencies and private-sector partnerships leverages the Government's mission-related research, producing general-purpose, broadly useful, and interoperable technologies, tools, and applications. That makes the NITRD Program a powerful engine of technology transfer. The large number of Federally funded IT breakthroughs subsequently commercialized in the private sector - often by g raduates of U.S. research universities whose education was supported by NITRD funding - leverage the Federal investments even further. The following pages describe the IT research priorities and plans of the NITRD agencies.