Internal replication as a tool for evaluating reproducibility in preclinical experiments

TL;DR

This paper proposes a framework to assess reproducibility in preclinical experiments by leveraging internal replication within existing data, offering a cost-effective alternative to full replication studies.

Contribution

It introduces six types of internal replication and demonstrates how to quantify and test reproducibility using multi-site mice data, enhancing robustness in preclinical research.

Findings

Internal replication can effectively estimate reproducibility.

Quantitative methods for internal reproducibility are demonstrated.

Framework improves robustness of statistical inferences.

Abstract

Reproducibility is central to the credibility of scientific findings, yet complete replication studies are costly and infrequent. However, many biological experiments contain internal replication, which is defined as repetition across batches, runs, days, litters, or sites that can be used to estimate reproducibility without requiring additional experiments. This internal replication is analogous to internal validation in prediction or machine learning models, but is often treated as a nuisance and removed by normalisation, missing an opportunity to assess the stability of results. Here, six types of internal replication are defined based on independence and timing. Using mice data from an experiment conducted at three independent sites, we demonstrate how to quantify and test for internal reproducibility. This approach provides a framework for quantifying reproducibility from existing data and reporting more robust statistical inferences in preclinical research.