Reinventing High Performance Computing: Challenges and Opportunities

TL;DR

The paper discusses the transformative shifts in high-performance computing driven by geopolitical, economic, and technological changes, emphasizing the need for innovative approaches to future HPC system design.

Contribution

It highlights the evolving landscape of HPC, emphasizing the importance of end-to-end co-design, custom hardware, and collaborations with cloud and ecosystem companies for future advancements.

Findings

Global shifts in HPC leadership and funding sources.

Emergence of cloud systems as new HPC paradigms.

Need for rethinking HPC design fundamentals.

Abstract

The world of computing is in rapid transition, now dominated by a world of smartphones and cloud services, with profound implications for the future of advanced scientific computing. Simply put, high-performance computing (HPC) is at an important inflection point. For the last 60 years, the world's fastest supercomputers were almost exclusively produced in the United States on behalf of scientific research in the national laboratories. Change is now in the wind. While costs now stretch the limits of U.S. government funding for advanced computing, Japan and China are now leaders in the bespoke HPC systems funded by government mandates. Meanwhile, the global semiconductor shortage and political battles surrounding fabrication facilities affect everyone. However, another, perhaps even deeper, fundamental change has occurred. The major cloud vendors have invested in global networks of massive scale systems that dwarf today's HPC systems. Driven by the computing demands of AI, these cloud systems are increasingly built using custom semiconductors, reducing the financial leverage of traditional computing vendors. These cloud systems are now breaking barriers in game playing and computer vision, reshaping how we think about the nature of scientific computation. Building the next generation of leading edge HPC systems will require rethinking many fundamentals and historical approaches by embracing end-to-end co-design; custom hardware configurations and packaging; large-scale prototyping, as was common thirty years ago; and collaborative partnerships with the dominant computing ecosystem companies, smartphone, and cloud computing vendors.