ExaGRyPE: Numerical General Relativity Solvers Based upon the Hyperbolic PDEs Solver Engine ExaHyPE

TL;DR

ExaGRyPE is a suite of numerical relativity solvers built on ExaHyPE 2, employing high-order finite difference schemes, adaptive mesh refinement, and parallel computing to simulate black hole spacetimes efficiently.

Contribution

The paper introduces ExaGRyPE's software architecture, domain-specific Python interface, and its application to black hole simulations, advancing numerical relativity tools for exascale computing.

Findings

Successfully simulated gauge wave, black holes, and binary systems.

Demonstrated the code's maturity and usability.

Identified numerical challenges for future research.

Abstract

ExaGRyPE describes a suite of solvers for numerical relativity, based upon ExaHyPE 2, the second generation of our Exascale Hyperbolic PDE Engine. The presented generation of ExaGRyPE solves the Einstein equations in the CCZ4 formulation under a 3+1 foliation and focuses on black hole spacetimes. It employs a block-structured Cartesian grid carrying a higher-order order Finite Difference scheme with adaptive mesh refinement, it facilitates massive parallelism combining message passing, domain decomposition and task parallelism, and it supports the injection of particles into the grid as data probes or tracers. We introduce the ExaGRyPE-specific building blocks within ExaHyPE 2, and discuss its software architecture and compute-n-feel: For this, we formalize the creation of any specific simulation with ExaGRyPE as a sequence of lowering operations, where abstract logical tasks are successively broken into finer tasks until we obtain an abstraction level that can be mapped onto a C++ executable. The overall program logic is fully specified via a domain-specific Python interface, we map this logic onto a more detailed set of numerical tasks, subsequently lower this representation onto technical tasks that the underlying ExaHyPE engine uses to parallelize the application, before eventually the technical tasks are mapped onto task graphs including the actual PDE term evaluations, initial conditions, boundary conditions, and so forth. We end up with a rigorous separation of concerns which shields ExaGRyPE users from technical details and hence simplifies the development of novel physical models. We present the simulations and data for the gauge wave, static single black holes and rotating binary black hole systems, demonstrating that the code base is mature and usable. However, we also uncover domain-specific numerical challenges that need further study by the community in future work.