A Comparison of Reproducibility Guidelines and Its Implications on Undergraduate Statistical Education

TL;DR

This paper examines the challenges of reproducing scientific research through a replication of a Bayesian study on broadband access and online course enrollment, highlighting implications for undergraduate statistics education.

Contribution

It demonstrates the difficulties in research reproduction despite open data and code, and discusses how reproducibility practices should influence undergraduate teaching.

Findings

Replication revealed challenges despite open materials

Implicit researcher assumptions complicate reproduction

Highlights need for improved reproducibility education

Abstract

In this paper, we replicated a Bayesian educational research project, which explores the association between broadband access and online course enrollment in the US. We summarized key findings from our replication and compared them with the original project. Based on my replication experience, we aim to demonstrate the challenges of research reproduction, even when codes and data are shared openly and the quality of the materials on GitHub are high. Moreover, we investigate the implicit presumptions of the researchers' level of knowledge and discuss how such presumptions may add difficulty to the reproduction of scientific research. Finally, we hope this article sheds light on the design of reproducibility criterion and opens up a space to explore what should be taught in undergraduate statistics education.