GR-Athena++: magnetohydrodynamical evolution with dynamical space-time

TL;DR

GR-Athena++ is a new, high-performance code for simulating magnetohydrodynamics in dynamical spacetime, enabling accurate modeling of astrophysical phenomena like neutron star mergers and black hole formation.

Contribution

It introduces a GRMHD code with dynamical spacetime capabilities built on Athena++, incorporating Z4c formulation and adaptive mesh refinement for scalable, stable simulations.

Findings

Successfully verified with benchmark problems involving neutron stars and black hole formation.

Accurately captures magnetic field instabilities with minimal divergence errors.

Demonstrates high scalability on supercomputers, enabling large-scale astrophysical simulations.

Abstract

We present a self-contained overview of GR-Athena++, a general-relativistic magnetohydrodynamics (GRMHD) code, that incorporates treatment of dynamical space-time, based on the recent work of (Daszuta+, 2021)[49] and (Cook+, 2023)[45]. General aspects of the Athena++ framework we build upon, such as oct-tree based, adaptive mesh refinement (AMR) and constrained transport, together with our modifications, incorporating the Z4c formulation of numerical relativity, judiciously coupled, enables GRMHD with dynamical space-times. Initial verification testing of GR-Athena++ is performed through benchmark problems that involve isolated and binary neutron star space-times. This leads to stable and convergent results. Gravitational collapse of a rapidly rotating star through black hole formation is shown to be correctly handled. In the case of non-rotating stars, magnetic field instabilities are demonstrated to be correctly captured with total relative violation of the divergence-free constraint remaining near machine precision. The use of AMR is show-cased through investigation of the Kelvin-Helmholtz instability which is resolved at the collisional interface in a merger of magnetised binary neutron stars. The underlying task-based computational model enables GR-Athena++ to achieve strong scaling efficiencies above $80\%$ in excess of $10^5$ CPU cores and excellent weak scaling up to $\sim 5 \times 10^5$ CPU cores in a realistic production setup. GR-Athena++ thus provides a viable path towards robust simulation of GRMHD flows in strong and dynamical gravity with exascale high performance computational infrastructure.