Openness and Reproducibility: Insights from a Model-Centric Approach

TL;DR

This paper offers a model-centric, probability-informed analysis of openness and reproducibility, clarifying key concepts and delineating the components necessary for scientific reliability and auditability.

Contribution

It introduces a conceptual framework based on idealized experiments to clarify the relationship between openness and reproducibility in science.

Findings

Identifies components of experiments needed for reproducibility and auditability.

Provides a formal, probability-based framework for understanding scientific reliability.

Clarifies distinctions between reliability, auditability, and reproducibility.

Abstract

This paper investigates the conceptual relationship between openness and reproducibility using a model-centric approach, heavily informed by probability theory and statistics. We first clarify the concepts of reliability, auditability, replicability, and reproducibility--each of which denotes a potential scientific objective. Then we advance a conceptual analysis to delineate the relationship between open scientific practices and these objectives. Using the notion of an idealized experiment, we identify which components of an experiment need to be reported and which need to be repeated to achieve the relevant objective. The model-centric framework we propose aims to contribute precision and clarity to the discussions surrounding the so-called reproducibility crisis.