Benchmarking with Supernovae: A Performance Study of the FLASH Code

TL;DR

This paper evaluates the performance and energy efficiency of the FLASH astrophysics simulation code on new high-performance computing hardware, comparing Intel, AMD, and ARM-based systems for simulating supernovae.

Contribution

It provides a comprehensive benchmarking of FLASH on diverse modern hardware architectures, highlighting optimal configurations for performance and energy efficiency in supernova simulations.

Findings

Intel Sapphire Rapids CPUs offer high performance for FLASH.

ARM-based A64FX processors show competitive energy efficiency.

Optimal MPI configurations improve simulation scalability.

Abstract

Astrophysical simulations are computation, memory, and thus energy intensive, thereby requiring new hardware advances for progress. Stony Brook University recently expanded its computing cluster "SeaWulf" with an addition of 94 new nodes featuring Intel Sapphire Rapids Xeon Max series CPUs. We present a performance and power efficiency study of this hardware performed with FLASH: a multi-scale, multi-physics, adaptive mesh-based software instrument. We extend this study to compare performance to that of Stony Brook's Ookami testbed which features ARM-based A64FX-700 processors, and SeaWulf's AMD EPYC Milan and Intel Skylake nodes. Our application is a stellar explosion known as a thermonuclear (Type Ia) supernova and for this 3D problem, FLASH includes operators for hydrodynamics, gravity, and nuclear burning, in addition to routines for the material equation of state. We perform a strong-scaling study with a 220 GB problem size to explore both single- and multi-node performance. Our study explores the performance of different MPI mappings and the distribution of processors across nodes. From these tests, we determined the optimal configuration to balance runtime and energy consumption for our application.