Building a Culture of Reproducibility in Academic Research

TL;DR

This paper discusses practical strategies for fostering a reproducibility culture in academic research, emphasizing self-interest, social processes, and standardized tools to improve research transparency and reliability.

Contribution

It offers a personal, experience-based framework for building reproducibility culture, highlighting actionable lessons and emphasizing the role of self-interest and social processes.

Findings

Reproducibility efforts benefit those who invest in them.

Standardized tools and social processes are key to fostering reproducibility.

Self-interest motivates researchers to adopt reproducibility practices.

Abstract

Reproducibility is an ideal that no researcher would dispute "in the abstract", but when aspirations meet the cold hard reality of the academic grind, reproducibility often "loses out". In this essay, I share some personal experiences grappling with how to operationalize reproducibility while balancing its demands against other priorities. My research group has had some success building a "culture of reproducibility" over the past few years, which I attempt to distill into lessons learned and actionable advice, organized around answering three questions: why, what, and how. I believe that reproducibility efforts should yield easy-to-use, well-packaged, and self-contained software artifacts that allow others to reproduce and generalize research findings. At the core, my approach centers on self interest: I argue that the primary beneficiaries of reproducibility efforts are, in fact, those making the investments. I believe that (unashamedly) appealing to self interest, augmented with expectations of reciprocity, increases the chances of success. Building from repeatability, social processes and standardized tools comprise the two important additional ingredients that help achieve aspirational ideals. The dogfood principle nicely ties these ideas together.