Status of reproducibility and open science in hep-lat in 2021

TL;DR

This paper surveys the reproducibility of lattice field theory research in 2021, highlighting progress and areas needing improvement in making computational results fully reproducible.

Contribution

It provides the first comprehensive assessment of reproducibility practices in hep-lat submissions, identifying strengths and gaps in open science adoption.

Findings

High reproducibility in some research areas

Opportunities for increased transparency and data sharing

Most submissions lack detailed reproducibility documentation

Abstract

As a fully computational discipline, Lattice Field Theory has the potential to give results that anyone with sufficient computational resources can reproduce, going from input parameters to published numbers and plots correct to the last byte. After briefly motivating and outlining some of the key steps in making lattice computations reproducible, this contribution presents the results of a survey of all 1,229 submissions to the hep-lat arXiv in 2021 of how explicitly reproducible each is. Areas where LFT has historically been well ahead of the curve are highlighted, as are areas where there are opportunities to do more.