Terminologies for Reproducible Research

TL;DR

This paper examines how different disciplines use the terms 'reproduce' and 'replicate' inconsistently in the context of reproducible research, aiming to clarify terminology and reduce confusion across fields.

Contribution

It provides a classification of disciplinary terminologies regarding reproducibility and replication, highlighting contradictions and proposing patterns to resolve terminological conflicts.

Findings

Disciplinary terminologies either conflate or distinguish 'reproduce' and 'replicate'.

Two main camps exist: one equates reproduce with same data/methods, the other with independent confirmation.

Terminology inconsistencies exacerbate reproducibility challenges.

Abstract

Reproducible research---by its many names---has come to be regarded as a key concern across disciplines and stakeholder groups. Funding agencies and journals, professional societies and even mass media are paying attention, often focusing on the so-called "crisis" of reproducibility. One big problem keeps coming up among those seeking to tackle the issue: different groups are using terminologies in utter contradiction with each other. Looking at a broad sample of publications in different fields, we can classify their terminology via decision tree: they either, A---make no distinction between the words reproduce and replicate, or B---use them distinctly. If B, then they are commonly divided in two camps. In a spectrum of concerns that starts at a minimum standard of "same data+same methods=same results," to "new data and/or new methods in an independent study=same findings," group 1 calls the minimum standard reproduce, while group 2 calls it replicate. This direct swap of the two terms aggravates an already weighty issue. By attempting to inventory the terminologies across disciplines, I hope that some patterns will emerge to help us resolve the contradictions.