Reproducibility of density functional approximations: how new functionals should be reported

TL;DR

This paper emphasizes the importance of reproducibility in density functional approximations (DFAs) in computational chemistry, proposing standardized verification methods and advocating for open source code to ensure consistent and reliable functional implementations.

Contribution

It introduces a framework for verifying DFA reproducibility, including reference energy generation methods and the necessity of sharing source code for new functionals.

Findings

Inconsistent DFA implementations lead to different total energies across software.

Converged numerical parameters are essential for sub-micro Hartree energy accuracy.

Open source code of reference implementations is crucial for reproducibility.

Abstract

Density functional theory is the workhorse of chemistry and materials science, and novel density functional approximations (DFAs) are published every year. To become available in program packages, the novel DFAs need to be (re)implemented. However, according to our experience as developers of Libxc [Lehtola et al, SoftwareX 7, 1 (2018)], a constant problem in this task is verification, due to the lack of reliable reference data. As we discuss in this work, this lack has lead to several non-equivalent implementations of functionals such as BP86, PW91, PBE, and B3LYP across various program packages, yielding different total energies. Through careful verification, we have also found many issues with incorrect functional forms in recent DFAs. The goal of this work is to ensure the reproducibility of DFAs: DFAs must be verifiable in order to prevent reappearances of the abovementioned errors and incompatibilities. A common framework for verification and testing is therefore needed. We suggest several ways in which reference energies can be produced with free and open source software, either with non-self-consistent calculations with tabulated atomic densities or via self-consistent calculations with various program packages. The employed numerical parameters -- especially, the quadrature grid -- need to be converged to guarantee a $\lesssim0.1\mu E_{h}$ precision in the total energy, which is nowadays routinely achievable in fully numerical calculations. Moreover, as such sub-$\mu E_{h}$ level agreement can only be achieved when fully equivalent implementations of the DFA are used, also the source code of the reference implementation should be made available in any publication describing a new DFA.