Virginia Polytechnic Institute and State University

Cost-Conscious Supercomputing

Logistics

Priced Right, Right Off the Shelf

In addition to its processing speed, the new Macs met two other crucial criteria: pricing and availability. For Virginia Tech, it was important to create the new supercomputer exclusively with off-the-shelf components. And, as with any project in higher education, budget was a concern. The Power Mac G5 computers ably addressed both considerations.

“This project never would have been possible at this price, while getting this performance, with any other [platform].”

— Jason Lockhart, Director of the College of Engineering’s High-Performance Computing and Technology Innovation Group, Virginia Tech

Says Lockhart, “In addition to our requirements to have a 64-bit, high-bandwidth memory and communications platform, we were committed to using [off-the-shelf] components from start to finish. We knew that a system of this size will have failures, and we wanted to minimize the expense of replacing failed components.

“Also, while a similar system built in Japan cost $350 to $400 million for the machine alone, and other moneys for facilities and personnel ran their total up to one billion, we only had $4 to $5 million to spend,” Lockhart adds. “The Madison generation of Intel’s Itanium2 chips was really expensive — just one processor was as much as $7,500! But by going with the Power Mac G5 computers — because they were cost-effective, they needed minimal tweaking (aside from adding some additional RAM), and they would scale really well — we were able to hit our budget and meet all of our performance goals.”

No Time to Spare

Developing a supercomputer is a colossal undertaking under any circumstances, and normally requires several years. For Virginia Tech, the goal seemed almost impossible: The system had to be complete and fully functional before October 1, 2003 — less than three months from the project’s start date. At that time, researchers from Germany’s University of Mannheim, the University of Tennessee, and Lawrence Berkeley National Laboratory would publish their semiannual “TOP500 List” of supercomputers, ranking them by processing speed.

Winning a berth near the top of the list would qualify Virginia Tech to compete for funding from the National Science Foundation (NSF), through its Cyberinfrastructure program. With a proposed budget of $1 billion, the program aims to develop and support an enhanced cyberinfrastructure for science and engineering research and education.

“The Cyberinfrastructure program has been ramping up with what they call the ‘Extensible Terascale Facility,’ or the distributed terascale facility,” explains Lockhart. “Basically, it’s a large-scale computational network that’s being established across the U.S., between large supercomputer centers. The NSF has been supplying funding to national labs and supercomputer centers to develop resources based on processing speeds of one trillion floating point operations per second, or FLOPS.

“One of our original goals,” continues Lockhart, “was to position the university to compete in a much larger research arena. We felt that the combination of the Cyberinfrastructure funding and the supercomputing technology would enable us to bring in the ‘big science’ types of research projects.”

Apple’s Developer Relations helped with the hardware and software development. With this assistance from Apple, as well as other volunteers at Virginia Tech, the cluster surpassed the 10 TFLOP mark in time to qualify as third fastest in the world.

Introduction 1 2 3 4 5 6 7 8 9

Download this story