Nyx-RT: Adaptive Ray Tracing in the Nyx Hydrodynamical Code

TL;DR

Nyx-RT introduces an adaptive, GPU-optimized ray tracing method integrated with the Nyx code, enabling efficient, large-scale cosmological simulations of reionization with validated accuracy and scalability.

Contribution

It presents a novel adaptive ray tracing algorithm compatible with Exascale architectures, including source merging and overlap correction, integrated into the Nyx hydrodynamics code.

Findings

Achieved scalable performance on 1024 nodes and 4096 GPUs.

Validated convergence in reionization history and Lyman-alpha forest flux.

Demonstrated efficient simulation of cosmological reionization processes.

Abstract

Numerical methods for radiative transfer play a key role in modern-day astrophysics and cosmology, including study of the inhomogeneous reionization process. In this context, ray tracing methods are well-regarded for accuracy but notorious for high computational cost. In this work, we extend the capabilities of the Nyx N-body / hydrodynamics code, coupling radiation to gravitational and gas dynamics. We formulate adaptive ray tracing as a novel series of filters and transformations that can be used with AMReX particle abstractions, simplifying implementation and enabling portability across Exascale GPU architectures. To address computational cost, we present a new algorithm for merging sources, which significantly accelerates computation once reionization is well underway. Furthermore, we develop a novel prescription for geometric overlap correction with low-density neighbor cells. We perform verification and validation against standard analytic and numerical test problems. Finally, we demonstrate scaling to up to 1024 nodes and 4096 GPUs running multiphysics cosmological simulations, with 4096^3 Eulerian gas cells, 4096^3 dark matter particles, and ray tracing on a 1024^3 coarse grid. For these full cosmological simulations, we demonstrate convergence in terms of reionization history and post-ionization Lyman-alpha forest flux.