GEAR-RT: Towards Exa-Scale Moment Based Radiative Transfer For Cosmological Simulations Using Task-Based Parallelism And Dynamic Sub-Cycling with SWIFT

TL;DR

GEAR-RT introduces a novel particle-based radiative transfer solver in SWIFT, leveraging task parallelism and sub-cycling to significantly improve efficiency for cosmological simulations.

Contribution

It presents the first implementation of a particle-based FVPM radiative transfer method with sub-cycling in a task-parallel framework, enhancing performance and flexibility.

Findings

Sub-cycling reduces simulation runtime by over 90%.

GEAR-RT successfully validates standard radiative transfer tests.

The Ivanova et al. (2013) FVPM method is unsuitable for co-moving particles.

Abstract

The development and implementation of GEAR-RT, a radiative transfer solver using the M1 closure in the open source code SWIFT, is presented, and validated using standard tests for radiative transfer. GEAR-RT is modeled after RAMSES-RT (Rosdahl et al. 2013) with some key differences. Firstly, while RAMSES-RT uses Finite Volume methods and an Adaptive Mesh Refinement (AMR) strategy, GEAR-RT employs particles as discretization elements and solves the equations using a Finite Volume Particle Method (FVPM). Secondly, GEAR-RT makes use of the task-based parallelization strategy of SWIFT, which allows for optimized load balancing, increased cache efficiency, asynchronous communications, and a domain decomposition based on work rather than on data. GEAR-RT is able to perform sub-cycles of radiative transfer steps w.r.t. a single hydrodynamics step. Radiation requires much smaller time step sizes than hydrodynamics, and sub-cycling permits calculations which are not strictly necessary to be skipped. Indeed, in a test case with gravity, hydrodynamics, and radiative transfer, the sub-cycling is able to reduce the runtime of a simulation by over 90%. Allowing only a part of the involved physics to be sub-cycled is a contrived matter when task-based parallelism is involved, and is an entirely novel feature in SWIFT. Since GEAR-RT uses a FVPM, a detailed introduction into Finite Volume methods and Finite Volume Particle Methods is presented. In astrophysical literature, two FVPM methods are written about: Hopkins (2015) have implemented one in their GIZMO code, while the one mentioned in Ivanova et al. (2013) isn't used to date. In this work, I test an implementation of the Ivanova et al. (2013) version, and conclude that in its current form, it is not suitable for use with particles which are co-moving with the fluid, which in turn is an essential feature for cosmological simulations.