Using Jupyter for reproducible scientific workflows

TL;DR

This paper demonstrates how integrating domain-specific software with Jupyter notebooks enhances reproducibility, interactivity, and control in computational science workflows through two case studies in magnetism and algebra.

Contribution

It presents novel integrations of domain-specific tools with Jupyter, enabling high-level control, reproducibility, and interactive exploration in scientific workflows.

Findings

Improved reproducibility of computational workflows.

Enhanced interactivity and control over simulations.

Facilitated sharing and reuse of research outputs.

Abstract

Literate computing has emerged as an important tool for computational studies and open science, with growing folklore of best practices. In this work, we report two case studies - one in computational magnetism and another in computational mathematics - where domain-specific software was exposed to the Jupyter environment. This enables high-level control of simulations and computation, interactive exploration of computational results, batch processing on HPC resources, and reproducible workflow documentation in Jupyter notebooks. In the first study, Ubermag drives existing computational micromagnetics software through a domain-specific language embedded in Python. In the second study, a dedicated Jupyter kernel interfaces with the GAP system for computational discrete algebra and its dedicated programming language. In light of these case studies, we discuss the benefits of this approach, including progress toward more reproducible and reusable research results and outputs, notably through the use of infrastructure such as JupyterHub and Binder.