NGI Implementation Plan Appendix E: Department of Energy

	Executive Summary Goal 1: Network Technologies Related Research 1.1:  Introduction 1.2:  Network Growth Engineering 1.3:  End-to-End Quality of Service Related Research 1.4:  Security Related Research Goal 2: Advanced Network Testbeds 2.1:  Testbed Relevant Infrastructure 2.2:  Connectivity Goal 3: Revolutionary Applications 3.1:  Revolutionary Applications
Executive Summary	Department of Energy  FY 1999 - FY 2000  NGI Program   With over 20,000 nationwide users of dozens of DoE unique experimental facilities and high performance computing resources, DoE has a long history of successful network research, advanced network deployment, and advanced applications support -- the three NGI research areas. DoE is eager to join with other agencies participating in NGI to complement its own efforts. DoE's strategy for its FY 1999 to FY 2000 NGI activities is to leverage its current core programs in network and application research to enhance the Department's ability to satisfy mission requirements through advanced technologies such as distributed computing, national collaboratories, remote access to facilities, and remote access to petabyte-scale datasets with complex internal structure. This will be accomplished by experimenting with and integrating applications and network research technologies on multiagency advanced NGI testbeds. These testbeds will include DoE laboratories, universities, and other Federal research centers. DoE NGI network research will focus on developing network-aware middleware and application friendly tools and capabilities for its applications, as well as continuing research in high speed end system interfaces, network management, and differentiated services. The objective of this research is to enable more efficient and smarter use of network resources, as well as to support higher speeds (that is, end-to-end capacity).   This program is built on DoE's long and successful history of network research in high speed end-system interfaces, protocols and services to support collaborative environments, congestion and flow control, and management tools and techniques that has enabled the agency to achieve its mission activities and become a major contributor to the Federal government's Large Scale Networking Interagency Working Group. This program is also built on the system level integration expertise DoE has gained as a first adopter of advanced technologies. These enable the Department to effectively integrate advanced technologies from other Federal agencies, academia, and industry with DoE research efforts to support advanced applications.   University participation in DoE programs has traditionally focused on support for the university researcher, relying on the use of existing infrastructure such as the vBNS and ESnet (Energy Sciences network). However, successful incorporation of NGI technologies into DoE applications will require not only the vertical integration of the application, network research, and testbed areas, but also horizontal integration of different cultures (for example, CIOs with researchers) in order to align the research programs (researchers) with the infrastructure (networks and computing facilities) at both DoE labs and universities. In order to develop this horizontal integration, DoE will support a DoE-university partnership program that will be aimed at enhancing the collaboration among DoE and university researchers, technologists, CIOs, and infrastructure providers. This partnership will focus on providing advanced network capability from end to end (that is, from the campus to the DoE facility) so that the researchers at both locations attain the level of application collaboration they require to work on DoE's mission-critical programs.   In order to accomplish this goal, DoE will also support joint DoE-university network research to develop the necessary capabilities and tools required by the applications and infrastructure administrators at DoE labs and select universities, as well as deploy DOE2000 tools and capabilities to support critical DoE mission applications. DoE will support the researchers at both the labs and the universities, to enable them to adapt their DoE application codes to make use of these new technologies as they are being developed (for example, DOE2000 services such as Class Based Queuing (CBQ) and Public Key Infrastructure (PKI), and to work with the network researchers to ensure that the new technologies are responsive to their application requirements. DoE will also support enhancements of certain "critical path" infrastructure elements such as ESnet, aggregation and interconnection points (for example, GigaPOPs), and local networks and services, to implement and support these new technologies to provide the appropriate level of end-to-end services to the application. Network management and analysis tools that function across networks and administrative boundaries and deal with these new capabilities will be developed.   DoE will coordinate its basic research program activities and infrastructure with other agencies' NGI activities when such coordination (1) supports DoE's basic mission and (2) enhances the overall NGI program. In return, DoE will benefit from close coordination with activities of other NGI agencies to ensure the early adoption and rapid insertion of new NGI technologies into DoE applications. DoE will also ensure that ESnet peers and interconnects with other agency networks in order to provide access to DoE's facilities and enable cross agency collaborations in support of DoE's missions.   DoE's programmatic and mission focused network research are applicable to NGI Goals 1, 2, and 3 during FY 1999 to FY 2000. These efforts are further described in the remainder of this appendix.
Goal 1	Goal 1:  Network Technologies Related Research
1.1 Introduction	Introduction   DoE will extend its LSN network technologies research in areas that support application access to and use of Quality of Service (QoS) capabilities at the network, operating system, and middleware layers. These activities will focus on providing the necessary "network aware" and infrastructure manipulating software in middleware, including libraries, system software, and tools that will be available to the application through easy-to-use application interfaces. DoE will increase its efforts to enable effective application control of Internet Protocol (IP), Asynchronous Transfer Mode (ATM), other media (for example, WDM), implementations of multicast, and network management capabilities. DoE will also enhance its efforts to develop ultra high speed end system interfaces and protocols (for example, HIPPI-64 ST) to provide DoE applications with the end-to-end performance they require. This will be accomplished through the development of middleware, tools, and interfaces that provide DoE applications access to and control of efficient (for example, operating system bypass) ultra high speed media. These Goal 1 services and mechanisms will be incorporated into DoE's ESnet and other appropriate NGI testbeds early in the development process to enable DoE's applications to immediately benefit from the most recent networking improvements.   Quality of Service and network management capabilities are two critical requirements for enabling DoE's advanced distributed and collaborative applications. DoE will continue its development of CBQ and other class of service (CoS) and QoS mechanisms required by its demanding applications. These capabilities will be incorporated into DOE2000 and NGI middleware and libraries. In order to further enhance the middleware services and capabilities made available to DoE applications, DoE will explore methods to provide IP, ATM, and WDM resource and admission control, scheduling, management, prioritization, accounting (such as bidding and costing), authentication, analysis, monitoring, assurance, and debugging mechanisms in application-friendly network-aware middleware. DoE will also investigate how to maximize its use of these new and advanced services and technologies through the concurrent support of both network research and production traffic on the same infrastructure (for example, Goal 2.1 100x networks).   DoE's work in intelligent and network-aware middleware, network growth engineering, QoS, and security will be coordinated with DARPA and other agencies to ensure that the advances made in these areas to support DoE application requirements will compliment the work that others are pursuing to address their application requirements. Specifically, DoE's activities will concentrate on providing the "network aware" middleware support required by DoE applications, which will be heavily collaborative in nature and will concurrently use distributed resources such as supercomputers, high end storage systems with extremely large scientific data sets, unique on-line facilities, and massive, multidimensional datasets in tele-immersive environments.
1.2 Network Growth Engineering	Network Growth Engineering   1.2.1  Monitoring, Control Analysis, and Display   DoE will continue its research on IP, ATM, and WDM network monitoring and analysis tools (for example, PATHchar and Network Probe Daemon (NPD)), but with the explicit purpose of providing appropriate application level tools, APIs, and middleware support for gathering relevant network status so that the applications can dynamically adjust their use of the underlying infrastructure.   1.2.2  Data Delivery   DoE will evolve it current support of IP, ATM, and WDM based congestion and flow control techniques and mechanisms, as well as multicast capabilities, to provide applications with easy-to-use tools, capabilities, and interfaces that make efficient use of advanced infrastructure (for example, reliable ordered multicast).   1.2.3  Managing Lead User Infrastructure   DoE will investigate and deploy an architecture that supports both network research and production (advanced application) traffic on as much of the same infrastructure as possible. This will enable the applications to concurrently stress new technologies and take advantage of advanced preproduction capabilities without undergoing massive transitions. It will also support the dynamic creation and use of virtual networks required by the numerous multi-site DoE collaborations, as well as provide the application and system level administrative tools and capabilities required to manage such an environment.   DoE will also investigate the extension of QoS capabilities and "striped" network access into the operating system (workstation, parallel systems, storage servers) and end system architecture, as well as the middleware and libraries that provide the application interface, to provide for efficient application to low level service plane control and framing (for example, direct application control of WDM wavelengths). In addition, DoE will complete its R&D on multi-gigabit ultra high speed end system interfaces, analyzers, and switches (for example, HIPPI64) as well as on developing mechanisms (for example, HIPPI-64 ST) to reduce operating system overhead for data transfers. Middleware support will be developed (for example, DOE2000 Advanced Communications Technology Satellite (ACTS) libraries) to provide application access to these capabilities. Monitoring, Control, Analysis, and Display   FY 1999   Application level interfaces that leverage PATHchar, NPD, RED, and other appropriate tools, and that will convey network status to DoE's applications, are developed.   FY 2000 APIs and tools are integrated into DOE2000 and NGI middleware to provide a base capability to all DoE applications. Data Delivery   FY 1999 Reliable and ordered multicast capabilities are developed and tested.   FY 2000 User friendly interface and API for multicast is integrated into DOE2000 and NGI middleware. Managing Lead User Infrastructure   FY 1999 Initial capabilities to support both dynamically created short and long term collaborative virtual networks are integrated into ESnet.   FY 1999 Network management enhancements are made to WDM based switches and an API is developed.   FY 1999 HIPPI 64 switch developed   FY 1999 Scheduled Transfer (ST) operating systems bypass capabilities developed   FY 1999 Intelligent middleware platform (for example, DOE2000 ACTS, NEXUS, and POOMA) supports CBQ, DOE2000 security "use" rules, and prototypical network status gathering capabilities.   FY 1999 Solicitation issued for proposals from joint university and DoE lab network researchers to develop required intelligent middleware, APIs, and tools for application use of and control of network infrastructure for high performance end-to-end applications. Awards made.   FY 2000 APIs are integrated into DOE2000 and NGI middleware so that applications can initiate the creation of the virtual networks.   FY 2000 A DoE application is ported to an ultra high speed WDM testbed to demonstrate application control of WDM infrastructure.   FY 2000 ST and VIA capabilities merged and supported in middleware   FY 2000 Intelligent middleware is enhanced to provide applications dynamic control over creation of virtual networks, as well as provide general integrated resource management (that is, access to network, processors, data, on-line facilities) API and capabilities.   FY 2000 Extend support for dynamically created and long term collaborative virtual networks to select university and peering point infrastructures to support the new DoE-university partnership program.   FY 2000 Develop and deploy new intelligent middleware and network management technologies and capabilities across ESnet, peering points, and both lab and university campus infrastructures.
1.3 End-to-End QoS Related Research	End-to-End Quality of Service Related Research   1.3.1  Baseline QoS Architecture   DoE will develop and deploy network admission control, scheduling, management, prioritization, accounting (such as bidding and costing), authentication, analysis, monitoring, assurance, and debugging mechanisms that will support DoE application use of QoS. These mechanisms will be supported in intelligent "network aware" middleware layer that provides application controlled Class of Service (CoS) and QoS, as well as to enhance the systems management and integration tasks associated with IP, ATM, and other technology networks. The goal is to develop, enhance, incorporate, and integrate as many of these new technologies into DoE's ESnet and experimental networks on an end-to-end (that is, application to application) basis as quickly as possible.   DoE will also develop QoS and CoS APIs that provide for semantic mapping of QoS from the application perspective to that provided by the underlying services, as well as to provide for cross-layer signaling and triggering of QoS mechanisms when necessary. This API will support DoE's Accelerated Strategic Computing Initiative (ASCI) and DOE2000 applications. In particular, CBQ, which is being developed for DOE2000, will be deployed in ESnet.   DoE will continue to coordinate its R&D in CoS and QoS with DARPA (QUORUM program). DoE will also work with NSF, NASA, and appropriate universities to integrate CBQ and other QoS/CoS advances into their networks, when appropriate, to support collaborative work on DoE's mission critical programs. Baseline QoS   FY 1999   Implement and support CBQ and Bandwidth Broker (BB) on ESnet.   FY 2000 Support cross domain/network (that is, ESnet to a GigaPOP) CBQ and BB, as well as provide an API for CBQ/BB in DOE2000 middleware. Develop end-system/site administrative CBQ/BB policy management capabilities.   FY 2000 Develop new network aware middleware (for example, Globus, Nexus), as well as adapt and integrate existing middleware (that is, ACTS) to provide DoE applications with the capability to make use of QoS.
1.4 Security Related Research	Security Related Research   Security is essential to the success of DoE programs such as ASCI and DOE2000, as well as to the NGI. It is needed to support secure and fair user access to and use of network resources (for example, CBQ/BB), support smart network management, provide secure inter-network peering (for example, surety of routing updates), perform accounting/costing, and provide access to on-line facilities (DOE2000 role based access support). A PKI that is integrated into and interacts with industry PKI is essential.   DoE will coordinate its relevant security R&D with DARPA, NASA, and NSF. Security   FY 1999   Deploy initial DOE2000 "role/use" based access control mechanisms to support remote access to DoE's unique on-line facilities and collaborative environments. This includes a DoE wide PKI.   FY 1999 Integrate authentication and security mechanisms into CBQ, as well as develop and deploy admission control mechanisms into ESnet to support DOE2000 and ASCI.   FY 1999 Develop and prototype a secure software distribution system (SSDS) to monitor system software versions and patches and notify the user or administrator what software needs to be upgraded.   FY 2000 Extend CBQ/BB and DoE security mechanism support (that is, policy management support and PKI) to select universities and peering points (for example, GigaPOPs) to provide end-to-end support for DoE-university based collaborations on DoE mission programs.   FY 2000 Extend the DOE2000 use and access rules to support general distributed resource management (that is, processors, network, on-line facilities, data, etc.)
Goal 2	Goal 2:  Advanced Network Testbeds   2.1 Testbed Relevant Infrastructure   DoE will continue to provide enhanced network infrastructure and capabilities to its mission critical programs and applications and will coordinate interconnection and peering mechanisms with NGI networks when necessary to satisfy the requirements of DoE applications and to provide programmatically-justified access to DoE's unique on-line facilities. DoE's ESnet network will provide the required enhanced connectivity to DoE mission related sites and coordinate peering arrangements with other NGI networks consistent with the DoE mission.   DoE will also work with and interconnect with the other Federal agency mission related networks as required by the DoE mission. These interconnections will be made at the speeds and locations, as well as media (that is, IP, ATM, WDM), required by the DoE mission and in accordance with the various Acceptable Use Policies (AUPs) for the networks.   As part of its DoE-university partnership program, and in order to achieve a new level of integration between the DoE and university research environments to support work on DoE mission critical applications, DoE will aid in the upgrade and enhancement to select peering points and university campus infrastructures to provide the end-to-end capabilities required by the application.   2.2 Connectivity NGI members.  Connectivity among NGI connection members and non-NGI DoE principal sites will be achieved through a variety of interconnections, depending on mission and performance requirements. Connections will be made almost exclusively by using peering interconnects with other agency NGI networks; however, ESnet will continue to connect to FIXs, NAPs, and other connection peering points as needed. GigaPOPs.  A number of universities will be forming collaboration groups with the equivalent of a regional network established to interconnect the members. Typically, these regional networks will establish one or more peering points for external connectivity, generally called GigaPOPs. DoE may establish direct peering and physical connections to select GigaPOPs when there is an overriding DoE mission requirement that cannot be satisfied via ESnet-vBNS interconnections or other interagency interconnections. Federal agencies.  Direct interconnects with other Federal agency networks have been used for a long time and will continue to evolve to satisfy agency requirements. Since the NASA NREN and ESnet reside on the same ATM infrastructure, interconnects between them can be established as virtual connections. These connections are shared whenever possible and are efficient. Similarly, direct interconnects with the vBNS are being implemented. A 45 Megabit per second (T3) interconnect in San Diego between ESnet at General Atomics and the vBNS at San Diego Supercomputing center (SDSC) has been established, and a second interconnect at the ATM level is planned at the Perryman, Maryland POP (point of presence). DoE major sites.  DoE national laboratories and collaboratory sites are the major ESnet members. ESnet already has a well developed, high performance interconnect structure established to serve mission requirements. Incremental upgrades to the ESnet infrastructure may be needed to meet the leading edge requirements and traffic demands of DoE programs and applications (for example, ASCI, DOE2000). National carriers.  Existing interagency networks, as well as the NGI, will require carrier-to-carrier interconnects among their respective level-2 (data link) and level-3 communications facilities to provide a seamless network environment to federally supported applications. Available standards do not yet support such commercial-quality interconnects for layer 2. This can be managed in the short term by providing intermediate L2 facilities among the carriers. At the Washington, DC NGI Exchange Point, DoE will provide one such ATM interconnect between the Sprint Connecticut Avenue POP and the MCI-based Perryman POP. International.  Although not directly addressed in the NGI, international connections already exist in the major Federal networks. DoE will continue to work with other Federal agencies to make the best use of international links consistent with the DoE mission; however, the selection of peering points and interconnection sites will be based on support for DoE applications. Testbeds   FY 1999   Solicit proposals for select DoE programs that are located at both labs and university and that require enhanced performance and capabilities in the lab, ESnet, peering point (GigaPOP), and campus infrastructures.   FY 1999 Upgrade the DoE labs (for example, two or three sites) that need OC-12 to provide appropriate level of performance for DoE as well as DoE-university collaborations.   FY 1999 Enhanced network interconnection and peering management capabilities are deployed.   FY 1999 Enhanced interagency NGI peering points (such as ATM and IP)   FY 2000 DOE2000 (for example, CBQ) and other appropriate NGI infrastructure performance upgrades across DoE and university infrastructures are in place and being used by the applications.   FY 2000 Implement enhancements for concurrent support (that is, MORPHNET) of both production and network R&D traffic across the lab, ESnet, peering point, and campus infrastructures.   FY 2000 Support QoS and inter-network policy management (such as QoS, security, network management) across networks and peering points.
Goal 3	Goal 3:  Revolutionary Applications   DoE applications that require the technologies and infrastructure outlined in this appendix are largely components of the DOE2000 initiative, which aims to provide DoE scientists and engineers with advanced collaboration technologies to make DoE's unique on-line facilities and resources more accessible to labs and universities. The DOE2000 initiative has three components: Advanced Computational Testing and Simulation (ACTS) Advanced computational methods that facilitate the application of scientific models to experimental, environmental, and simulation data. The ACTS User Toolkit will provide a science-friendly package of algorithms, software, interfaces, and other tools that make the power of high performance computers accessible to researchers. National collaboratories Laboratories without walls that unite expertise, instruments, and computers, enabling scientists to carry out cooperative research independent of geography. Collaboration tools will include networked file and database facilities, resource locators, video conferencing, remote instrumentation, multiple supercomputer software, and shared visualization and virtual reality applications. Pilot projects Virtual laboratories that give scientists the technology to collectively observe and attack problems using combinations of ideas, methods, and instruments that do not exist at any single location. The pilot projects include The Materials MicroCharacterization Collaboratory and The Diesel Combustion Collaboratory. Each of these components of DOE2000, as well as other DoE mission critical programs such as ASCI place significant demands on the other components to the NGI initiative to enable the distributed computing, remote access to and operation of facilities, and effective access to massive data resources that they require to be successful. In particular, the problem of effective access to tera- and petabyte scale data resources with sufficient interactivity to enable scientific progress requires significant advances in all of the NGI components as well as significant research in new paradigms to explore such data and technologies to store and process the data.   In order to achieve a new level of integration between the DoE and university research environments that support work on DoE mission critical applications, DoE will initiate a DoE-university partnership program that will support network research, accelerated infrastructure, and network-aware DoE applications. A solicitation for proposals will be issued asking for one or two strategic DoE programs that are collocated at and distributed across the DoE Labs and universities, coupled with a joint DoE-university application supportive network research and testbed infrastructure. It is expected that three to eight universities will be selected to participate. The network research will leverage DoE's efforts in the DOE2000 program (for example, CBQ and PKI) and its LSN core network research program, as well as LSN network research supported by other agencies. Revolutionary Applications   FY 1999   Issue a solicitation for proposals seeking one or two strategic DoE programs that are collocated at and distributed across the DoE labs and universities, that require enhanced network performance and support for a DoE mission critical application. Make the award.   FY 2000 Adapt the DoE mission critical application code and supporting infrastructure at the university and the DoE lab to use the required DOE2000 and NGI technologies.   FY 2000 Adapt a revolutionary DoE application to take advantage of an ultra high speed WDM network and its capabilities.