To provide an infrastructure to support science and technology research, NSF, DOE, NASA, NIH, NOAA, and EPA fund high end computing and communications facilities enabling scientists across the country to run large-scale applications at these sites via remote connections. This permits scientists to:

• Evaluate early prototype systems and provide feedback to developers
• Integrate visualization and virtual reality into high performance systems
• Run full-scale applications, including Grand Challenge and other breakthrough applications, on systems not otherwise available
• Develop parallel software using scaled down systems

Researchers at these facilities rely on enabling technologies, including high- speed networks, supercomputing systems, often with parallel processor architectures, massive data storage, and virtual reality display devices. Multidisciplinary research teams, including university faculty, facility managers and staff, hardware and software vendors, and industrial affiliates, contribute to the facilities' overall success; IT R&D funding is leveraged through equipment and personnel from vendors, discipline-specific agency funds, state and local funds, and industrial affiliate contributions. Industrial affiliation and outreach activities offer a low-risk environment for exploring and exploiting IT R&D technologies.

Applications software developers access these facilities over the Internet and experimental networks such as the NGI. All facilities provide extensive undergraduate educational opportunities and training for researchers, graduate students, and faculty members, and publish professional journal articles, annual reports, and newsletters. Most also offer K-12 educational programs.

The following systems--excluding workstation clusters-are funded by the named agencies and may receive additional funding from other IT R&D agencies. For example, funding for systems at NSF centers also comes from DARPA, NASA, and NIH.

The Partnerships for Advanced Computational Infrastructure (PACI) program supports the National Computational Science Alliance (Alliance), headquartered at the University of Illinois at Urbana-Champaign (UIUC), and the National Partnership for Advanced Computational Infrastructure (NPACI) at the San Diego Supercomputer Center (SDSC). Each of these leading-edge sites maintains a variety of high end computing systems and supports, individually or in tandem, more than 60 geographically distributed partner institutions from 29 states that maintain smaller systems.

Taken as a whole, the PACI partnerships constitute a large, distributed computing environment connected via high-speed networks, over which the partners contribute to the infrastructure by developing, applying, and testing the necessary software and tools to drive further growth of this "national grid" of interconnected high performance computing systems. PACI provides the foundation for meeting the expanding need for high end computation and information technologies required by the U.S. academic community, supporting and developing information-intensive uses of the computing systems available on the grid and the technologies needed to support their use.

The partnerships provide:

• Access to a diverse set of advanced and mid-range computing engines, data storage systems, and experimental architectures
• Enabling technologies, by developing both software tools for parallel computation and software to access the partnership's widely distributed, architecturally diverse machines and data sources and effectively exploiting the partnership's very large distributed systems
• Application technologies, by allowing researchers in high end applications to develop and optimize discipline-specific codes and software infrastructures to make these available to the program as a whole and to researchers in other areas
• Education outreach and training, building an understanding of how to use high performance computing and communications resources and broadening the base of participation to ensure the Nation's continued world leadership in computational science and engineering

National Center for Supercomputing Applications (NCSA) at UIUC

• Twelve SGI Origin 2000s with an aggregate of 1,528 processors (768 at 500 Mflops peak each and 760 at 390 Mflops peak each), 680 Gflops peak, 618 GB memory, 4.3 TB disk storage
• NT Supercluster with 128 550 MHz Pentium III processors (550 Mflops peak each) and 32 330 MHz Pentium II processors (330 Mflops peak each), 81 Gflops peak, 144 GB memory
• HP/Convex Exemplar SPP-2000 with 64 180 MHz PA 8000 processors, 46 Gflops peak, 16 GB memory

• Four SGI Origin 2000s with an aggregate of 192 195 MHz R10000 processors, 75 Gflops peak, 24 GB memory
• Four SGI Power Challenges with an aggregate of 42 processors and 6.2 GB memory

• HP/Convex Exemplar SPP-2200 with 64 200 MHz PA 8200 processors, 51 Gflops peak, 16 GB memory

• Los Lobos Linux Supercluster with 512 733 MHz Pentium II processors, 375 Gflops peak, 256 GB memory (new in 2000)
• Roadrunner Linux Supercluster with 128 450 MHz Pentium II processors, 57.6 Gflops peak, 32 GB memory
• IBM SP2 with 96 66 MHz Power2 processors, 25 Gflops peak, 6 GB memory

• IBM SP with 200 222 MHz Power3 processors, 178 Gflops peak, 100 GB memory (new in 2000)
• IBM SP with 192 160 MHz Power2 processors, 123 Gflops peak, 100 GB memory

• Condor flock-pool of approximately 400 machines of various types
  
  

San Diego Supercomputer Center

• IBM RS/6000 SP with 1,152 222 MHz Power3 processors, 1.02 Tflops peak, 576 GB memory-the most powerful computing platform available to the U.S. academic community (new in 2000)
• Cray T3E with 272 300 MHz Alpha 21164 processors, 154 Gflops peak, 34 GB memory
• IBM SP with 128 160 MHz Power2 processors, 82 Gflops peak, 32 GB memory
• Sun HPC10000 with 64 400 MHz UltraSparc II processors, 51 Gflops peak, 64 GB memory
• Cray T90 with 14 processors, 24 Gflops peak, 4 GB memory
• Tera MTA with 16 processors, 8 Gflops peak, 16 GB memory (new in 2000)

• Cray T3E with 88 300 MHz Alpha 21164 processors, 34 Gflops peak, 11 GB memory
• Cray SV1 with 16 processors, 19.2 Gflops peak, 16 GB memory University of Michigan
• IBM SP with 64 160 MHz Power2 processors, 31 Gflops peak, 64 GB memory

University of Michigan

• IBM SP with 64 160 Mz Power2 processors, 31 Gflops peak, 64 GB memory

• HP Exemplar X-Class with 256 processors, 184 Gflops peak, 64 GB memory
• HP V2500 with 128 processors, 128 GB memory (new in 2000)
  
  

National Center for
Atmospheric Research
(NCAR)

• IBM SP with 144 dual-processor (200 MHz Power3) nodes (128 compute nodes), 204 Gflops peak, 128 GB memory, 2.5 TB disk
• Compaq ES40 with 8 four-processor nodes, 32 Gflops peak, 32 GB memory
• Two Cray J924se/1024s with 24 processors, 8 GB memory, 1.5 Gflops sustained total
• Cray J920/512 with 20 processors, 4 GB memory, 60 Mflops/processor sustained
• Cray J916/256 with 16 processors, 2 GB memory, 60 Mflops/processor sustained
• SGI Origin 2000/128 with 128 250 MHz R10000 processors, 16 GB memory
• SGI Origin 2000/16 with 16 250 MHz R10000 processors, 16 GB memory
  
  

NASA testbeds and
facilities

NASA maintains testbeds throughout the country to offer diversity in configuration and capability. The testbeds include:

Numerical Aerospace Simulation (NAS) Facility, NASA Ames Research Center, Moffett Field, California

• SGI Origin 2000 with 512 CPUs, 192 GB memory
• SGI Origin 2000 with 256 CPUs, 64 GB memory
• SGI Origin 2000 with 64 CPUs, 16 GB memory
• SGI Origin 2000 with 24 CPUs, 7 GB memory
• Cray C90 with 16 CPUs, 8 GB memory
• SGI Origin 2000 with 12 CPUs, 3 GB memory (for long-term storage)
• Cray C90 with 7 CPUs, 2 GB memory

NASA Glenn Research Center, Cleveland, Ohio

• SGI Origin 2000 with 24 CPUs, 6 GB memory

NASA Goddard Space Flight Center, Greenbelt, Maryland

• Cray T3E-600, 1,296 processors, 162 GB memory
• Cray J932se, 32 processors, 8 GB memory
• Cray SV1, 24 processors, 8 GB memory

Jet Propulsion Laboratory, Pasadena, California

• SGI Origin 2000, 128 processors (R12000, 300 MHz), 32 GB memory, 6 TB disk
• Cray SV1-1A, 16 processors, 8 GB memory, 480 GB disk

NASA Langley Research Center, Langley, Virginia

• SGI Origin 2000 with 16 CPUs, 4 GB memory
  
  

DOE laboratories

The NERSC facility at LBNL in Berkeley, California, provides production computing for investigators supported by the Office of Science as well as researchers at universities and Federal laboratories. Some of these resources are available through a peer review allocation process. Research areas include high-energy and nuclear physics, fusion energy, materials sciences, chemistry, life sciences, environmental sciences, Earth and engineering sciences, and applied mathematics and computational science. NERSC's resources, with a total theoretical speed of 1.2 Tflops, include:

• Cray T3E-900 with 692 processors, 623 Gflops peak, 177 GB memory, and 2.8 TB disk storage
• IBM RS/6000 SP with 608 Power3 200 MHz processors, 486 Gflops peak, 256 GB memory, and 10 TB disk storage
• Cray PVP cluster consisting of three Cray SV1s and a Cray J90se, with a total of 96 vector processors, 4 gigawords (GW) memory, and a peak performance of 83 Gflops
• Two HPSS tertiary storage systems with 1,000 TB total capacity
• SGI Onyx 2 server for scientific visualization from remote locations
• Parallel Distributed Systems Facility (PDSF), a networked, distributed system of 53 workstations and eight disk vaults with file servers, supporting large-scale computation for high-energy and nuclear physics investigations
  
  

Two kinds of high performance systems in the Advanced Computing Laboratory (ACL) at LANL support applications and power the ACL's software infrastructure:

• "Nirvana," an SGI/Cray system consisting of 16 shared memory multiprocessors, each with 128 processors and 32 GB of memory. The system interconnect is HiPPI 800 technology. Nirvana has 6.91 TB of disk storage with aggregate peak bandwidth of 4.8 GBps. This 2,048-processor ensemble machine provides 1 Tflop (theoretical), making it one of the highest-capability unclassified computer systems in the world. With 10 SGI Infinite Reality graphics engines that enable interactive visualization and analysis previously unavailable, Nirvana is the largest graphics supercomputer in the world. Nirvana supports applications for the Office of Science, universities in the DOE Accelerated Strategic Computing Initiative's (ASCI's) Academic Strategic Alliance Program (ASAP), and LANL institutional programs.
• A system of experimental, low-cost clusters of machines running Linux is supported for applications and research into making such clusters more effective. Currently, the largest of these is "Rockhopper," with 128 dual-CPU nodes (500 MHz Pentium IIIs), each with 1 GB of memory and 9 GB of disk storage. The system interconnect is Myrinet, with 14 usec latency and 1.2 GBps bandwidth. The system provides 128 Gflops peak performance and has shown per-processor performance parity with Nirvana on some applications.
  
  

The high end systems at ORNL include an IBM SP-3 and a Compaq Alphaserver with a total theoretical speed of 1.5 Tflops/s and 360 TB HPSS storage, dedicated to unclassified scientific computing:

• "Eagle" is an SMP cluster with 184 nodes, each with four IBM Power3 II (1.5 Gflops) processors, 1.08 Tflops/s theoretical speed, 372 GB memory, and 9.2 TB local storage
• "Falcon" is an SMP cluster with 80 nodes, consisting of four Compaq EV67 (1.3 Gflops) processors, 427 Gflop/s theoretical speed, 160 GB memory, and 5.5 TB local storage Together, they form a high end production system and applications testbed for atmospheric radiation, biology, chemistry, climate, combustion, materials, nanotechnology, many-body physics, and spallation neutron source design.

PNNL's high end systems have a total theoretical speed of 414 Gflops with 48 TB of EMASS storage. Uses include applied mathematics, atmospheric sciences, biology, chemistry, climate modeling, engineering, environmental molecular sciences, natural resources management, and subsurface reactive transport.

• "NWMPP1" is an MPP IBM SP with 512 0.5 Gflops processors, 247 Gflops theoretical speed, 262 GB memory, and 5.0 TB local storage

NIH computing systems

The Center for Information technology (CIT) has a 46-processor SGI Origin 2000 parallel computer and a 224-processor Beowulf cluster. Both systems and other high-performance computing resources are used by the NIH scientific staff in biomedical applications. NCI's Frederick Biomedical Supercomputing Center has a 96-processor Cray SV1 supercomputer, a 64-processor SGI Origin 2000 parallel computer, and a collection of biomedical software that are available to scientists who use the facility. The National Center for Research Resources (NCRR) supports various systems for biomedical research applications at its six High Performance Computing Resources Centers, which include:

• Resource for Concurrent Biological Computing, Beckman Institute, University of Illinois
• Supercomputing for Biomedical Research, Pittsburgh Supercomputing Center
• Theoretical Simulation of Biological Systems, Columbia University
• Parallel Computing Resource for Structural Biology, University of North Carolina, Chapel Hill
• Biomedical Computation Resource, University of California, San Diego
• Parallel Processing Resource for Biomedical Scientists, Cornell Theory Center, Cornell University

NCRR also supports two Scientific Visualization Resource Centers:

• Interactive Graphics for Molecular Studies, University of North Carolina, Chapel Hill
• Special Research Resource for Biomolecular Graphics, University of California, San Francisco

NOAA laboratories

NOAA operates two high end computing centers that work closely with the computer science community to advance the art of programming highly parallel scalable systems for geophysical fluid dynamics problems:

• The Forecast Systems Laboratory in Boulder, Colorado, has a system from HPTi with 256 Compaq Alpha processors and 128 GB memory. This system is used to explore highly parallel regional and mesoscale forecast models under severe wall-clock constraints.
• The Geophysical Fluid Dynamics Laboratory (GFDL) in Princeton, New Jersey, has a Cray T-90 PVP and a 128-processor Cray T3E 900 MPP. These systems are used for the global climate modeling and weather forecasting Grand Challenges.
  
  

EPA systems

EPA's National Environmental Supercomputing Center (NESC) in Research Triangle Park, North Carolina, is dedicated to environmental research and problem solving for characterizing and quantifying risks to human health and to the integrity, resilience, and sustainability of ecosystems now and in the future. The NESC high performance computing resources include:

• Cray T3E-1200 with 72 processors and 18 GB memory, 76 Gflops peak
• Cray T3D with 128 processors and 8 GB memory, 19.2 Gflops peak
• Cray C94/3128 with 4 processors and 128 MW memory, 3 Gflops peak