The Fundamental Principles of Reproducibility

TL;DR

This paper clarifies the concept of reproducibility in science by analyzing the scientific method, surveying existing literature, and defining three types and four degrees of reproducibility, with machine learning as an example.

Contribution

It provides a fundamental, structured framework for understanding and classifying reproducibility in scientific research, addressing terminological confusion.

Findings

Defined three types of reproducibility

Specified four degrees of reproducibility

Applied framework to machine learning experiments

Abstract

Reproducibility is a confused terminology. In this paper, I take a fundamental view on reproducibility rooted in the scientific method. The scientific method is analysed and characterised in order to develop the terminology required to define reproducibility. Further, the literature on reproducibility and replication is surveyed, and experiments are modeled as tasks and problem solving methods. Machine learning is used to exemplify the described approach. Based on the analysis, reproducibility is defined and three different types of reproducibility as well as four degrees of reproducibility are specified.