High Performance Computing

References & Resources

---

"...once you do know what the question actually is, you'll know what the answer means."
— Douglas Adams (1952-2001), as the supercomputer "Deep Thought" in The Hitchhiker's Guide to the Galaxy

---

"High Performance Computing" is one of those terms that means many things to many people. At the most basic level, HPC tries to answer big questions: How do hurricanes work? Are aliens trying to contact Earth? What does protein "X" look like? Where is John Doe's record in a terabyte database? What happens if I push the trigger on this nuclear bomb? And no, the answer to the latter question is not "boom.." ;) HPC is the stuff of national laboratories, universities, and corporate research centers; it involves a myriad of concepts, including:

• Supercomputers
• Clusters (including the ever-popular Beowulf)
• Parallel programming
• Distributed computing
• Execution threads & concurrency
• Numerical methods

HPC is becoming increasingly important, both in the scientific and business communities. The vast server farms of Google, the clusters used to simulate nuclear explosions, or the computers involved in SETI@Home — all are examples of different approaches to solving big problems by applying various combinations of computers and software.

My professional programming career has always focused on the processing side of computing, the acts of number crunching and data manipulation, looking for the most effective methods of making hardware "sweat". Along the way, I've played with supercomputers, clusters, artificial life simulations, and vast databases. Recent events have rekindled my involvement in HPC, and so I've added this section to Coyote Gulch.

I've programmed parallel and  clustered computers in the past, but have never actually built such a system myself. So a budding Beowulf cluster, named Herodotus, is becoming reality in my workshop. Once the system is up and running the way I want, I'll be extending my explorations of Complexity to include parallel processing algorithms and truly autonomous agents. And I'll be sharing what I know, and what I learn. My goal is to provide an HPC resource similar in scope and flavor to what I've done with Complexity Central.

Standards & Technologies

Parallel Virtual Machine (PVM)
A standard for parallel program development.

Message Passing Interface (MPI) Forum
Official MPI standards documents, errata, and archives of the MPI Forum.

Message Passing Interface (MPI)
A standard library specification for message-passing, at Argonne National Laboratory.

OpenMP
Supports multi-platform shared-memory parallel programming in C/C++ and Fortran.

JavaSpaces
A unified mechanism for coordination of Java objects.

High Performance Virtual Machines (HPVM)
An effort to build high performance applications on distributed systems.

Object Oriented MPI (OOMPI)
An object oriented approach to MPI, for C++.

Parallel Programming Languages

The Occam Archives
Information relevant to the Occam programming language.

CODE
A visual parallel programming system that composes sequential programs into parallel programs.

Orca
A language for parallel programming on distributed systems, based on the shared data-object model

NESL
A parallel programming language that includes concepts from functional programming languages.

Fortran Parallel Programming Systems
A collection of resources for parallel programming in FORTRAN.

ZPL
An array programming language designed from first principles for fast execution on both sequential and parallel computers.

mpC
A high-level parallel language (an extension of ANSI C), for developing portable applications for heterogeneous networks.

V
Extends C with a new type constructor for vectors and introduces parallelism using an apply-to-each construct and built-in primitives.

Organizations

Partnerships for Advanced Computational Infrastructure (PACI)
An initiative of the U.S. National Science Foundation.

Center for Applied Scientific Computing (CASC)
Scientific HPC at Lawrence Livermore National Laboratories.

National HPC Software Exchange
Software, documents, data, and information of interest to the HPC and parallel computing.

National Energy Research Scientific Computing Center (NERSC)
Provides high-performance computing tools and expertise.

National Center for Supercomputing Applications (NCSA)
At the University of Illinois.

HiPerSoft
Fosters research on the design and implementation of HPC software.

Parallel Architecture Research Laboratory (PARL)
Parallel and distributed computing systems for science and engineering.

Edinburgh Parallel Computing Centre
Promotes the effective exploitation of high performance parallel computing systems.

Windows Clusters Resource Centre
Resources for developers and users of Windows NT/2000 PC computational clusters, at the University of Southampton, England.

Condor
A project to develop, implement, deploy, and evaluate HPC and distributed software mechanisms and policies.

Concurrent Systems Architecture Group (CSAG)
Research focused on hardware and software architecture issues in parallel and distributed computer systems

Netlib Repository
A collection of mathematical software, papers, and databases, at Oak Ridge National Laboratory.

Beowulf & Clustering

Linux Clustering Information Center
A central repository of links and information regarding Linux clustering, in all its forms.

High-Availability Linux Project
A community development effort promoting a high-availability (clustering) solution for Linux.

Cluster Computing Information Center
Lots and lots of links to cluster-oriented sites, software, and information.

Beowulf Underground
Unsanctioned and unfettered information on building and using Beowulf systems.

Scyld Beowulf
Commercial Beowulf from the people who invented it.

Debian - Beowulf
An official Debian distribution for Beowulf clusters

Beowulf Software
A compendium of Beowulf-related software packages.

Beowulf Home
Official Beowulf site; seems a bit out-of-date.

Windows 2000 Supercomputing
Microsoft's resources for Windows-based clusters.

Distributed Computing

distributed.net
Advocacy for distributed computing; numerical applications.

SETI@Home
Searching for extraterrestrial messages in radio signals.

Folding@home
Understanding self-assembly through protein folding.

Genome@home
Designing new genes for working proteins.

Climate Prediction
Global climate research at the Rutherford Appleton Laboratory.

United Devices
Distributed computing software and services.

PC Philanthropy
Intel's peer-to-peer, distributed computing initiative.

Misc. Tools

LAM/MPI
An MPI programming environment and development system.

Portable, Extensible Toolkit for Scientific Computation (PETSc)
A suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations.

Guide to Available Mathematical Software
A database of numerical tools at the National Institute of Science (NIST).

Publications

Journal of Parallel and Distributed Computing

Parallel and Distributed Computing Practices

Parallel Programming with MPI, a book by Peter Pacheco