Fundamental Computational Sciences
DOE's Oak Ridge and Los Alamos National Laboratories are using massively parallel computing systems for numerical simulations of quantum chromodynamics (QCD), the fundamental theory of the strong interactions of high energy physics. Such work can help unlock fundamental secrets of nature. For example, lattice QCD is a branch of elementary particle physics that seeks to further understanding of the properties of quarks and gluons (the most basic constituents of matter) by computational modeling of their interactions.
QCD software developed for several parallel systems (including the Cray Research T3D, IBM SP-2, Intel Paragon, Thinking Machines CM- 5, and workstation clusters) provides stress tests for all key system features -- integer and floating point operations, inter-node communications, and I/O. In one instance previously undetected problems were revealed, and large sections of code are now included in a vendor's diagnostic software.
http://physics.indiana.edu/~sg/qcd_doe.html
High Capacity Atomic-Level Simulations for the Design of Materials
The objective of this NSF-supported Grand Challenge at Caltech is to develop theoretical methods for practical computations of the structures and properties of real materials for use in industrial process design for manufacturing new materials. There is an enormous gap between current methods for atomistic simulations and the level needed to accurately describe the relevant properties of industrially important materials. The strategy is to transcend from the most fundamental theory, quantum mechanics, where simulations are limited to small systems (10 to 20 atoms), through molecular dynamics where systems of a few thousand atoms are possible, to new techniques suitable for practical chemical engineering software directed at complex molecules.
A potentially significant industrial application for atomistic simulations is predicting the glass temperature T of a polymer: above T the polymer is soft and can be formed, while below T it is stiff. Industry would like to adjust T to desired values by manipulating the polymer. This is done empirically today, leading to costly and wasteful experiments. Excellent predictions of T were recently obtained for a variety of cases (including Teflon) in a collaboration involving Chevron, BF Goodrich, and Asahi Glass. The mechanism controlling T was also explained via the simulations, suggesting that in the future it may be possible to control T by making specific changes to the polymer.
http://www.wag.caltech.edu
First Principles Simulation of Materials Properties
The work on the DOE-funded Grand Challenge in Materials Sciences is distributed among Oak Ridge National Laboratory, Brookhaven National Laboratory, and Ames Laboratory at Iowa State University. The effort is to use high performance computing resources to design materials. A key to designing material for structural, magnetic, optic, electrical, and high temperature applications is our understanding and ability to control synthesis and processing at the atomic level. As many of the crucial macroscopic properties of materials actually depend on defects, clusters, and microscopic structures involving hundreds to thousands of atoms, it is only with the availability of modern high performance computing systems that first principles modeling of these structures and related materials properties can be undertaken. To this end a hierarchy of increasingly accurate and computationally intensive techniques have been developed, tested, and are now being applied -- classical molecular dynamics, tight binding molecular dynamics, and ab initio methods. Parallel computing combined with new algorithms that scale linearly with the number of atoms are being used to calculate efficiently the electronic structure and quantum-mechanical forces for systems of up to 500 atoms for ab initio methods and up to 10,000 atoms for tight binding molecular dynamics using an Intel Paragon XP/S 35.
The melting and pressure-temperature phase diagram of carbon is an example in which conditions are too extreme for laboratory experiments, but in which accurate molecular dynamics simulations are leading to new insights for understanding natural and artificial diamond synthesis. The image below shows a snapshot of 512 carbon atoms in a diamond lattice in the process of melting (T > 4000K). The red atoms indicate four-fold bonded (diamond-like) atoms, the blue atoms indicate three-fold (graphitic) bonded atoms, and there are also a number of two-fold and five-fold coordinated atoms. The large number of three-fold atoms indicates that the liquid phase is less dense than the four-fold diamond phase. This is in contrast to silicon, in which the liquid phase has a higher average coordination than in the diamond structure. By running such simulations for the coexistence conditions of the solid and liquid phases, the melting temperatures of diamond as a function of pressure are determined.
The melting of diamond.
Magnetic alloys are at the heart of a wide range of technological applications from the oldest of structural materials to the next generation of data storage and retrieval devices. However, a detailed microscopic understanding of alloy magnetism is lacking, hindering further development of these technologies. Using a new ab initio method, researchers have been able to study, for the first time, the nature of the magnetic state in disordered alloys. In the image below, Ni- (large blue spheres) and Cu- (small red spheres) atoms occupy the lattice sites of a 256-atom/unit cell model of a Ni-rich disordered NiCu alloy. The local Ni-site magnetic moment is distributed inhomogeneously, varying from a minimum of approximately 0.29 Bohr magnetons (short blue arrows) to a maximum of approximately 0.6 Bohr magnetons (long red arrows). Interestingly, the magnetic moment on a Ni-site correlates with the total magnetic moment on the nearest neighbor shell of atoms surrounding it: large red arrows tend to be surrounded by other reddish arrows, while small blue arrows are surrounded by either Cu sites having no moment or other blue arrows. The results of these simulations are being used to re-interpret results of neutron- scattering measurements of magnetic correlations in these alloys and to provide new insights into the properties of magnetic alloys.
http://www.ccs.ornl.gov/GC/materials/MShome.html
The magnetic structure of a Ni-rich NiCu alloy.
Black Hole Binaries: Coalescence and Gravitational Radiation
Black holes are formed by pressureless dust. The three-dimensional spiraling coalescence of two black holes is a problem of fundamental importance in astrophysics and general relativity. Such an event would produce a strong source of gravitational radiation that will be detectable by LIGO (Laser Interferometer Gravitational- wave Observatory) by the turn of the century. Solving this problem requires using advanced computational techniques to solve the Einstein field equations. The solution is computationally intensive and requires new methods for data management and visualization. Adaptive gridding and multigrid techniques applied to hyperbolic and parabolic systems of equations can be used in other computationally intensive problems in science and engineering. Tackling such problems in turn will stimulate new developments in architectures and algorithms for massively parallel and vector systems. This NSF-funded Grand Challenge is centered at the University of Texas and involves a large number of collaborators from other institutions.
http://godel.ph.utexas.edu/Center/GC/page1.html
Scalable Hierarchical Particle Algorithms for Galaxy Formation and Accretion Astrophysics
This NASA Grand Challenge has the goals of (1) understanding structure formation on distance scales from sub-galactic to cosmological, and (2) studying accretion problems such as stellar collisions and disruptions and accretion onto a black hole. For these studies scalable parallel particle software -- N-body, smoothed- particle hydrodynamics (SPH), and hybrid -- based on hierarchical tree data structures is being implemented on the IBM SP-1, Intel Paragon, and Thinking Machines CM-5. Load balancing is achieved by (1) using the self-similar (Morton ordered) curve that traverses the volume of the simulated domain, (2) assigning particles corresponding to different pieces of the curve to different processors, and (3) assigning adjacent particles to the same processor, thereby minimizing communication between processors. This approach can be implemented to be independent of the nature of interparticle forces, making it applicable to similarly formulated problems in physics, chemistry, molecular biology, and engineering.
http://cesdis.gsfc.nasa.gov/hpccm/accomp/94accomp/94accomps.html
NASA has developed galaxy formation models that display different views of the same simulation.
Radio Synthesis Imaging
This NSF-funded effort at the University of Illinois is implementing prototype next-generation astronomical telescope systems Ñ remotely located telescopes connected by high speed networks to high performance computing systems and on-line databases accessed by astronomers over gigabit speed networks. The current prototype links the Berkeley-Illinois-Maryland Millimeter Array (BIMA) to NCSA for real-time data transfer into a database and for archiving on the NCSA mass storage system. Computationally intensive software for processing radio synthesis array data is being implemented on parallel systems. This work is applicable to other remotely controlled data-intensive facilities.
http://atlas.ncsa.uiuc.edu/hpcc-radioastro
Large Scale Structure and Galaxy Formation
This NSF-funded Grand Challenge is developing new parallel algorithms and software development strategies in order to use teraflops computing systems to answer two of the most fundamental questions in the physical sciences: What is the origin of large-scale structure in the universe, and how do galaxies form?
Cosmological simulations require computing the motion of millions of particles subject to their mutual gravitational attraction. To do this calculation efficiently on massively parallel computing systems is a challenge when the particles become highly clustered. It requires load balancing, that is, giving each processor an equal amount of work. A perfectly load-balanced algorithm that permits simulations with tens of millions of particles has been developed for the Thinking Machines CM-5.
The same algorithm has been applied to more earthly problems involving the motion of many mass particles -- two examples are the spreading of oil droplets after a marine oil spill and modeling fuel injection sprays in internal combustion engines. A modified algorithm that incorporates contact forces is being applied to the flow of food or drug pellets in a processing machine.
This work is being conducted by seven institutions that comprise GC3, the Grand Challenge Cosmology Consortium that is anchored by NCSA and PSC.
http://zeus.ncsa.uiuc.edu:8080/BlueBook96/BB96.html
Simulation of gravitational clustering of dark matter. This detail shows one sixth of the volume computed in a cosmological simulation involving 16 million highly clustered particles that required load balancing on a massively parallel computing system. Many particles are required to resolve the formation of individual galaxy halos seen here as red/white spots.