High performance communications activities are a highly productive HPCC Program area. During the past year we have seen continued explosive growth in the use of high performance communications for information access, research at the frontiers of science, and commercial endeavors. High performance communication activities include continued research and development in internetworking technologies to support these new communities and applications, research into network systems operating at a billion bits per second and faster, wireless technologies, integrating computing and communication across wide areas, and enhanced Internet connectivity.

Internetworking R&D

The Internet was created by ARPA, and that agency, other HPCC agencies, and industry and academia have contributed to its evolution and phenomenal growth (details about its evolution are given below). As new communities of education, commercial, public sector, and individual users join the Internet, fundamental changes must be made to support increased use, new applications such as video and imagery, service quality, security, and new modes of communication. Internetworking R&D supports these activities across the HPCC Program.

Work at ARPA, NSF, and other HPCC agencies in protocols is directed at ensuring that different types of networks continue working together as a cohesive system. New services, such as multicast and guaranteed service quality, are being created, evaluated, and introduced. Research into new routing and scaling techniques for extremely large networks anticipates an Internet that interconnects millions of sub-networks and billions of end users across an increasingly diverse set of media. Network security from the perspectives of protecting the network and protecting users is being incorporated into the fundamental operating principles.

• In FY 1995 ARPA will demonstrate network bandwidth, delay, and service reservation guarantees to support new services, such as video, imagery, collaborative work, and interactive data services. In conjunction with other DOD agencies, ARPA will test and evaluate SONET and ATM (described below) encryption technologies at 155 Mb/s (megabits per second or millions of bits per second). Also in FY 1995, next generation multicast-based services with improved ease of use will be demonstrated.

• In FY 1995 NSF plans to support integrative research, including experimental projects, that spans computer architecture, operating systems, compilers, I/O systems, network interfaces, and networks. The objective is to reduce latency and increase performance of the network integrated computing system.

• In FY 1995 DOE will support R&D and deployment to integrate scalable I/O with network R&D.

The Evolution of the Internet

The Internet began in the late 1960s with the development of ARPANET. ARPANET provided a limited number of researchers with shared, interactive communications between computing systems at different locations and developed key innovative technologies on which the Internet still depends. Some of these include:

• "Packet switching" in which a message is divided into packets that are transmitted to their destination and reassembled -- rather than using a dedicated circuit as in the telephone system or "store and forward" that is used in the telegraph system

• A distributed rather than a centralized network -- even when some nodes fail, the remaining network continues to function, although perhaps at reduced capability

• Adaptivity -- packets can be transmitted to the same destination over different parts of the network based on current network load and connectivity

• The Transmission Control Protocol (TCP) and the Internet Protocol (IP) were developed in the 1970s and early 1980s under ARPA sponsorship to enable different networks to interoperate. TCP/IP made it possible for ARPANET, the ARPA packet radio network, and the packet satellite network, each with different packet sizes and network transmission speeds, to interoperate.

While initially an experimental system, the need for a stable network emerged as the user community grew. MILNET was split from the ARPANET for use by Department of Defense users. Federal agencies established networks to support their R&D communities: NSF created NSFNET for the university research community, NASA built the NASA Science Internet (NSI), and DOE created the Energy Sciences Network (ESnet). These networks and an ever growing profusion of local area networks and regional networks form the HPCC component of the Internet, a "network of networks," and demonstrate the fundamental strength of new networking technologies created in the early ARPANET.

The recent phenomenal growth of the Internet beyond the HPCC community is the result of educational, public service, private sector, and personal investment. Today there are more than 27,000 networks in the U.S., an increase of 350 percent over that reported on these pages a year ago. Non-U.S. networks number more than 21,000, an increase of 225 percent. More than 1,500 U.S. colleges and universities, 1,500 high schools and 1,700 elementary schools in the U.S. have full Internet connectivity.

The HPCC Program has directly stimulated the emergence of a vigorous and highly competitive private sector industry in Internet hardware, software, and connectivity in which the U.S. is a world leader.

A major HPCC activity is the support of five gigabit testbeds jointly funded by ARPA and NSF with support from DOE through their national laboratories, and a sixth testbed funded by ARPA. Nine Federal agencies, thirteen telecommunications carriers, twelve universities, eight corporations, and two state supercomputer centers participate in these six testbeds that connect 24 sites. Transmission equipment and fiber optic cabling is provided by the carriers and industry at no cost to the Federal government. An important success of the gigabit networking project is the high level of cooperation among the academic, industrial, and governmental participants leading to new capabilities in the science community, new markets for the industrial community, and new visions for high performance communications.

http://WWW.CNRI.Reston.VA.US:4000/public/gigabit.html

HPCC-supported gigabit testbeds funded jointly by NSF and ARPA test high speed networking technologies and their application in the real world. Courtesy Corporation for National Research Initiatives.

One of the technologies further developed and deployed in the testbeds is Asynchronous Transfer Mode or ATM, a "fast packet switched" cell relay technology in which small packets of fixed 53- byte size are rapidly routed over a network. ATM is a technology being investigated by the HPCC Program and the U.S. telecommunications industry to integrate data, voice, and video services using a single protocol. "ATM/SONET" refers to transmitting ATM packets over a Synchronous Optical Network (SONET) link, a standard for transmitting data across fiber optic cable.

Some of the accomplishments of the gigabit testbed project are:

• Operation of high speed data transmission systems -- The testbeds used SONET systems to transmit data bits over optical fibers at speeds not previously achieved in long distance networks. Most testbeds operated at 2.4 billion bits per second (gigabits per second or Gb/s) across thousands of kilometers. A typical comprehensive encyclopedia contains about one billion bits.

• Transport protocols for wide-area networks -- The testbeds experimented with new very high-speed transport protocols. A number of testbeds used ATM as their technology for transport and switching. Others adapted the High Performance Parallel Interface (HiPPI), a standard interface for accessing supercomputers, to transport data between sites. New transmission records of 500-800 Mb/s between hosts over long distances were set with these transport protocols.

• ATM switches -- New ATM switches were developed to route data at high speed across the testbeds. Maximum network and switching speeds at the start of the project were T3 (45 Mb/s). The testbeds required higher speed switches and pushed switching to 155 Mb/s, 622 Mb/s, and faster.

• Host interfaces -- Computing systems vendors have delivered fast processors, but the development of fast communication interfaces to attached devices such as mass storage systems and to high-speed networks has lagged. Testbed researchers worked independently and with vendors to increase the speeds of host I/O and interfaces to the network. For example, one group demonstrated 400 Mb/s between two Digital Equipment Corporation (DEC) Alpha workstations using ATM/SONET interfaces.

• Distributed shared memory -- Network speeds increased, and improved host interface throughput enabled new approaches to sharing information across the network. Two testbeds explored implementations of a single virtual address space across multiple computers, significantly increasing the capability of a distributed computer system.

• Several groups of researchers developed prototype protocols to address network demands such as high speeds and additional performance requirements such as efficiently using network resources while providing different qualities of service guarantees needed by different types of applications. one group designed and demonstrated Tenet, a real-time protocol suite that delivers required quality of service bounds specified by the application, including delay jitter, throughput, and packet loss probability bounds. A second group developed (1) the Dynamic Time Windows protocol to avoid, and where necessary control, congestion for end- to-end flow control, and (2) Network Traffic Reporting that includes algorithms for measuring and controlling traffic and for scheduling and load control. A third group worked on cell-based integrated services at gigabit speeds to address similar problems using ATM.

• Several application environments were developed to facilitate using distributed computing systems. These include the Data Transfer Mechanism (DTM), Express, and Dome. DTM is a message passing facility to simplify interprocess communication and facilitate creating sophisticated distributed applications in a heterogeneous computing environment. Express is an integrated software package that addresses all phases of the application development cycle in an architecture-independent manner, including algorithm design, implementation, debugging, and performance analysis. The Dome (Distributed Object Migration Environment) project is developing libraries of distributed objects (that is, scalable software) for producing applications software for parallel execution on networks of computing systems.

• Applications -- All the testbeds included demanding applications that drive gigabit speed network development and provide criteria for evaluating effectiveness. These applications include chemical dynamics, global climate modeling, medical treatment planning, radio astronomy, and terrain visualization. Results from distributing select applications over multiple heterogeneous computing systems in the CASA testbed show superlinear increases in performance; that is, two computers working together solve the problem in less than half the time of either single computer.

CASA

CASA is a 2.4 Gb/s SONET testbed connecting the NSF San Diego Supercomputer Center (SDSC), the DOE Los Alamos National Laboratory (LANL), NASA's Jet Propulsion Laboratory (JPL), and California Institute of Technology (Caltech). Each of these sites has significant computing resources, and CASA focused on interconnecting these supercomputers. While LANL and the other sites are about 1,600 km apart, the link is 2,000 km long. This link holds world speed records of 500 Mb/s for TCP/IP file transfers and 792 Mb/s for raw HiPPI (High Performance Parallel Interface), a world record for host-to-host bandwidth-distance product of any ground-based testbed. The distance and speed presented new problems for network researchers. For example, in CASA there are over 800,000 bytes of data in flight at any instant. Any error in transmission causes unacceptable delay in processing. The CASA HiPPI/SONET gateway incorporates a forward error correction scheme to avoid re-transmission of data when an error is detected. The CASA testbed allows two groups of climate researchers, one in California and the other in New Mexico, each working on different parts of a problem, to combine their software models into a single metacomputer-based execution using heterogeneous computing systems.

http://www.noc.lanl.gov/lanp/project.html#casa

Irradiation of healthy tissue was reduced in an experimental radiation therapy medical treatment planning application that was distributed over a 622 Mb/s HiPPI/ATM/SONET network in the VistaNET testbed. Calculations were performed on a Cray Research Y-MP and displayed on a Pixel Planes workstation. This application provided data for real world network traffic analysis.

MAGIC

The MAGIC (Multidimensional Applications and Gigabit Internetwork Consortium) project was established to develop a very high speed, wide-area networking testbed to address challenges in heterogeneous computing, distributed storage, coordination of multiple data streams, and techniques for managing the effects of network delays in a defense applications context. The MAGIC team demonstrated TerraVision, a terrain visualization application that allows a user to view and navigate through a photorealistic landscape. With the addition of GPS-derived (GPS stands for Global Positioning System) data representing positions of units and vehicles, multiple commanders can get bird's eye views of situation and flow during training exercises as shown below. TerraVision images are stored on a distributed set of Image Server Systems (ISSs) connected by a high speed network. ISS images are transmitted to displays at Fort Leavenworth, KS, or Overland Park, KS. Sustained rates of several hundred Mb/s have been demonstrated.

http://www.magic.net/

To create a three-dimensional terrain scene in real time, distributed storage servers transmit data streams containing portions of the image in parallel over a high speed network to a user's workstation. In this view of a military training facility, the real-time vehicle position is shown by the blue line.

ATDnet

The Advanced Technology Demonstration Network (ATDnet) is a recently inaugurated high performance networking testbed in the Washington, DC area. It is intended to be representative of possible future Metropolitan Area Networks. Initially architected by NSA and the Naval Research Laboratory and funded by ARPA to enable collaboration among DOD and other Federal agencies, ATDnet has a primary goal to serve as an experimental platform for diverse network research and demonstration initiatives. ATDNet provides an opportunity for early access and familiarization for the DOD, especially those components dependent on state-of-the-art information technology. Emphasis is on early deployment, operation, and management of emerging ATM/SONET networks.

Six Federal agencies participate. ARPA and the Defense Information Systems Agency (DISA) serve as co-chairs of the ATDnet program. other members are the Defense Intelligence Agency (DIA), NASA, the Naval Research Laboratory (NRL), and NSA.

In FY 1995 ATDNet is used for experiments in telemedicine, distributed simulation, security and encryption techniques, ubiquitous video teleconferencing, large ATM network signaling research, ship design and analysis visualization, and network experiments. New network management techniques for SONET and ATM systems are being developed that will give DOD fast-start access to equivalent commercial services once they are tariffed.

http://www.disa.atd.net/disacfe/disacfe.html

ACTS

In FY 1994 NASA launched its Advanced Communications Technology Satellite (ACTS). ACTS provides a "network in the sky" and is capable of multiple low speed access, four 155 Mb/s full duplex connections, or a single 622 Mb/s full duplex connection. In FY 1995 NASA expects to demonstrate gigabit speed applications using ACTS in two experiments: "A Performance Study of an ATM/SONET Satellite and Terrestrial Network for an Engine Inlet Simulation," and "High Data ACTS Experiment for Performing Global Science: Keck (Hawaii) Telescope and Global Climate Model." ARPA and NASA will also demonstrate interconnection of terrestrial gigabit networks in FY 1995.

http://cesdis.gsfc.nasa.gov/hpccm/accomp/94accomp/ess94.accomps/ess3.html

http://kronos.lerc.nasa.gov/acts/experiment-program-overview-6.html

Additional gigabit networking R&D includes:

• Beginning in FY 1995 ARPA plans to develop advanced opto- electronic network component technology and network architecture for scalable and modular networks. The aggregate network bandwidth will be in the range of terabits per second, and the network will handle multimedia service for both digital and analog signals.

• ARPA and NSF will continue to co-manage the 10 Gb/s testbed at Washington University initiated under the Technology Reinvestment Program (TRP).

• In FY 1995 and FY 1996 NASA and DOE plan to demonstrate interoperability among independently managed ATM-based networks supplied by multiple vendors and used by multiple agencies for collaborative efforts. DOE will demonstrate constant bit rate services and other non-traditional data usage over ATM.

• NSA has a perennial requirement for the fastest networking technology in order to satisfy the National Security Grand Challenge. The goals of its networking efforts are to realize multi-gigabit per second trunking speeds and to support sustained data flows of at least hundreds of megabits per second. In the longer term, the work aims to achieve sustained data flows of many gigabits per second. In FY 1995 NSA will install an in-house optical crossbar network testbed; beginning in FY 1996 NSA will conduct R&D in high speed optical network architecture and optical network management techniques.

• In FY 1995 and FY 1996 NIST will collaborate with the ATM Forum to develop standardized conformity tests for ATM-based technology, and will collaborate with industry and users to establish an ATM testbed to test evolving products. NIST R&D includes simulation and design to assess interactions among reliability of data transmission, flow control, congestion control, and the right to transmit information, in order to reduce the likelihood of losing information during periods of high demand. Design flaws and ways to minimize data loss will be reported to developers and to appropriate standards organizations.

Wireless Technologies

As part of its program in National-Scale Information Enterprise, ARPA is extending the information infrastructure into the mobile environment. This includes wireless communications and networking technologies described here, and supporting services for mobile information systems (described in Section II.4):

• An environment is being implemented that allows a high density of in-building mobile users to simultaneously access multimedia computing systems and information servers connected to high bandwidth network backbones. In this environment a lightweight portable multimedia notebook called the InfoPad is used to access a pico-cell wireless network.

  http://infopad.eecs.berkeley.edu

• A design environment for the wireless networking environment is also being developed; a point-to-point wireless multimedia link using a PC notebook has been demonstrated, validating the overall design approach.

  http://millennium.cs.ucla.edu/wamis.html

• A high speed ATM-based wireless communication system is being developed that will be adaptive at both the network and the host interface levels. The system will allow for rapid deployment and response to a changing environment, thereby realizing the advantages of ATM for wireless networks.

  http://www.tisl.ukans.edu/RDRN/

In FY 1996 NSF will support basic research in wireless networks, including transmission, networking, and computing issues. The objective is to enable network access in a secure, seamless, and highly mobile environment, and to enable the handling of voice, video, and data communications.

In FY 1995 and FY 1996 NIST plans to assess the vulnerabilities and security requirements of emerging wireless technologies and to assess performance and establish benchmarks.

R&D for Network Integrated Computing

The computing systems and instruments used in research on the Grand Challenges and the National Challenges, as well as on Defense Challenges, are located throughout the country, and the researchers who use these resources travel around the world. One goal of the HPCC Program is to develop a networking and computing environment in which the number of researchers working on a problem and their locations are incidental to conducting the research. Another goal is to enable the extension of such an environment to an ever larger user community. Activities include the following:

• ARPA is developing technologies to move information efficiently among applications across widely distributed networks.

• ARPA is enabling dynamic configuration of networked computing resources; resource scheduling; toleration of failures peculiar to a wide area environment; toleration, avoidance, or hiding of unavoidable latency in a wide area network; and work with heterogeneous resources and across different administrative domains.

• In FY 1995 ARPA expects to integrate DOD and commercial networks and demonstrate crisis management technologies.

• DOE has on-going research in packetized video and voice, telecommuting, on-line facilities access and control, and collaborative workspaces. In FY 1995 DOE expects to release a version of its collaborative work technologies to other agencies for review and use and to complete packetization of commercial codex video/audio output that allows ESnet to carry packetized video encapsulated in IP (Internet Protocol). Such MBONE (multicast backbone) services are used in some 135 conferences or meetings each month and support NASA's multicast from Antarctica and the Space Shuttle as well as multicasting the Internet Engineering Task Force (IETF) and other Internet standards activities.

• In FY 1996 DOE plans to design and prototype protocols, techniques, and mechanisms to support enhanced network and workstation-based audio/video capabilities as well as virtual and electronic collaborative "lab spaces."

  http://www.mcs.anl.gov/home/stevens/labspace/root.html

• The DOE-funded ComPaSS (Communications Package for Scalable Software) project at Michigan State University addresses the design of communications libraries and supporting tools for distributed- memory computing platforms. A better understanding and implementation of collective communication for wormhole-routed networks, tools for data redistribution and resource management on workstation clusters, and host interface software for distributed computing across ATM networks from this work are evidenced in the incorporation of Michigan State's wormhole routing techniques into commercial switches as well as the development of analytical network tools to monitor and measure various application traffic implementing collective communications (multicast) over both raw ATM and TCP/IP over ATM.

  http://www.cps.msu.edu/~mckinley/ComPaSS/

Enhanced Internet Connectivity

At the beginning of the HPCC Program in 1991, NSF was upgrading the NSFNET backbone to T3 speed, and by 1992 full T3 backbone service was operational. This was the first network to provide T3 service. Today, as the NSFNET program prepares to switch over to a new architecture described below, commercial network service providers are offering T3 backbone service as are several regional networks. In fact all regional networks routinely provide T1 (1.5 Mb/s) service or better on their backbones.

As the Internet community grows, higher bandwidth and enhanced service networks are needed to handle more users, the increased size of transmitted objects (for example, images and video), and new applications such as collaborative work in which voice, video, and interactive data are transmitted. In the next year HPCC agencies will continue to enhance Internet capabilities. Examples are:

• By the end of FY 1995 NSF will complete the transition to a new architecture that consists of a set of Network Access Points (NAPs), funded by NSF, and accessed by network service providers (NSPs) typically at T3 speeds. NSPs will provide connectivity to the NAPs from the regional networks. A Routing Arbiter, also funded by NSF, will assist the cross network traffic at each NAP.

• In FY 1995 NSF will implement a very high speed Backbone Network Service (vBNS) to link the NSF Supercomputer Centers and NAPs at 155 Mb/s. NSF is issuing a solicitation in FY 1995 for provision of additional international Internet service at higher capacities than at present. Finally in FY 1995 NSF plans to announce a new Connections Program to support the use of innovative technologies for schools and libraries, to continue its regular program of funding connections for institutions of higher education, and to fund high bandwidth Internet connectivity for qualifying applications.

• DOE is expanding its ESNet with commercial ATM service in cooperation with NASA. DOE will upgrade ESNet to 45 Mb/s ATM services and will begin upgrading to 155 Mb/s at up to four sites in FY 1995. ESnet provides worldwide access to Energy Research f acilities including Light Sources, Neutron Sources, Particle Accelerators, Spectrometers, Genome Centers, and data bases.

  http://www.es.net/

• In FY 1995 NASA completed its upgrade to T3 connectivity and began its upgrade to 155 Mb/s among five NASA centers using a commercial ATM service: Ames Research Center at Moffet Field in Mountain View, CA; Goddard Space Flight Center in Greenbelt, MD; Langley Research Center in Hampton, VA; Lewis Research Center in Cleveland, OH; and the Jet Propulsion Laboratory (JPL) in Pasadena, CA. In FY 1996 NASA will demonstrate 622 Mb/s interoperation between Goddard and JPL.

• In FY 1995 at least 24 NOAA National Data Centers will be connected to the Internet through high bandwidth links; and in FY 1996 bandwidth will be increased to accommodate rapidly increasing user demand.

• NSF, the lead HPCC agency in connecting academic institutions, continues to fund connections for institutions of higher education at the rate of about 100 per year.

• At NIH, the National Library of Medicine's (NLM) Medical Connections program, in conjunction with NSF, provides "jump start" funding to academic medical centers, community hospitals, and other health care organizations to connect to the Internet. 170 U.S. medical schools and health care facilities have been connected during the past three years under the NSF and NLM Connections Program. Special emphasis is given to linking medical libraries with health care delivery organizations and database servers in support of timely and accurate clinical decision making. The program also supports the creation of regional consortia of health care institutions for sharing medical information and distribution of Internet capability within an institution. Approximately 15 grants are being awarded in FY 1995. The overall goal is to provide Internet connectivity to the top 3,000 health care institutions in the U.S.

  http://www.nlm.nih.gov/ep.dir/rfa_internet_conn.html

• The Department of Veterans Affairs (VA), an agency new to the HPCC Program in FY 1996, has an agency-wide network used to exchange medical and other data. This network is connected to the Internet via gateways, providing expanded functionality, wider access, and greater security. Additional capacity for transmitting multimedia data and using browsers such as Mosaic is being implemented.

• EPA has a dedicated T3 line between its National Environmental Supercomputing Center in Bay City, MI, and its National Computing Center and HPCC research network in North Carolina; this network facilitates technology transfer to state environmental groups.

• In FY 1995 NSF's International Connections Management Program (ICM) upgraded service to Stockholm and to London to 4 Mb/s each, and service to Paris to 2 Mb/s. Several additional Latin American and Caribbean countries are connected to or will presently connect to the ICM infrastructure. In the Pacific, Malaysia upgraded its connection from 64 Kb/s (kilobits per second or thousands of bits per second) to 1.5 Mb/s; at least two 64 Kb/s connections from the People's Republic of China were made; and Japan NACSIS upgraded its connection to 2 Mb/s. South Africa doubled its bandwidth from 128 Kb/s to 256 Kb/s. In all, about 40 countries are connected to the Internet by means of the ICM infrastructure and with the exception of the European connections, the connectivity is largely paid for by the other countries (NSF shares cost with the European countries on those connections).

• In FY 1995 DOE is upgrading links to Japan to 512 Kb/s, adding a 128 Kb/s link to Germany, installing data compressors on the link to Italy, and adding links to Russia, all in support of multi-national collaboration such as the International Thermonuclear Engineering Reactor.