Use Factorial Design To Improve Experimental Reproducibility

TL;DR

This paper advocates for using factorial experimental design to enhance reproducibility by simultaneously varying multiple factors, making it easier to identify sources of variability and improve experimental consistency.

Contribution

It demonstrates how factorial design methods can be integrated into routine experiments to better identify systematic sources of variability, addressing reproducibility issues.

Findings

Factorial design improves detection of variability sources.

Embedding factorial methods enhances experimental reproducibility.

Simple examples illustrate practical implementation.

Abstract

Systematic differences in experimental materials, methods, measurements, and data handling between labs, over time, and among personnel can sabotage experimental reproducibility. Uncovering such differences can be difficult and time consuming. Unfortunately, it is made more so when scientists employ traditional experimental procedures to explore possible sources of systematic variability by sequentially changing them one at a time to determine the magnitude of their effects. We use two simple examples to show how and why well known methods of factorial experimentation in which multiple potential sources are simultaneously varied provide a better alternative, can be understood as a straightforward extension of standard practice, and could be embedded into the quality control procedures of routine experimental practice. Doing so, we argue, would help mitigate at least some of the problems fueling the current "reproducibility crisis" roiling some scientific disciplines.