Perspectives on Reproducibility and Sustainability of Open-Source Scientific Software from Seven Years of the Dedalus Project

TL;DR

This paper reflects on seven years of developing the Dedalus open-source PDE solver, emphasizing the importance of open code policies for reproducibility and sustainability in scientific software, especially in astrophysics and related fields.

Contribution

It provides insights from long-term experience with Dedalus, advocating for comprehensive open-source policies including input files and analysis scripts to enhance reproducibility.

Findings

Open-source Dedalus has broad interdisciplinary use.

Reproducibility is hampered without open input files and scripts.

Seven years of development inform best practices for open scientific software.

Abstract

As the Science Mission Directorate contemplates establishing an open code policy, we consider it timely to share our experiences as the developers of the open-source partial differential equation solver Dedalus. Dedalus is a flexible framework for solving partial differential equations. Its development team primarily uses it for studying stellar and planetary astrophysics. Dedalus was developed originally for astrophysical fluid dynamics (AFD), though it has found a much broader user base, including applied mathematicians, plasma physicists, and oceanographers. Here, we will focus on issues related to open-source software from the perspective of AFD. We use the term AFD with the understanding that astrophysics simulations are inherently multi-physics: fluid dynamics coupled with some combination of gravitational dynamics, radiation transfer, relativity, and magnetic fields. In practice, a few well-known open-source simulation packages represent a large fraction of published work in the field. However, we will argue that an open-code policy should encompass not just these large simulation codes, but also the input files and analysis scripts. It is our interest that NASA adopt an open-code policy because without it, reproducibility in computational science is needlessly hampered.