ReproduceMeGit: A Visualization Tool for Analyzing Reproducibility of Jupyter Notebooks

TL;DR

ReproduceMeGit is a visualization tool designed to analyze and assess the reproducibility of Jupyter Notebooks in GitHub repositories, aiding researchers in identifying good and bad practices affecting reproducibility.

Contribution

It introduces a novel visualization and analysis tool that evaluates reproducibility of notebooks directly from GitHub repositories, integrating provenance export capabilities.

Findings

Identifies reproducibility success and failure rates in notebooks.

Provides detailed provenance information for reproducibility analysis.

Enables direct assessment of reproducibility practices in shared notebooks.

Abstract

Computational notebooks have gained widespread adoption among researchers from academia and industry as they support reproducible science. These notebooks allow users to combine code, text, and visualizations for easy sharing of experiments and results. They are widely shared in GitHub, which currently has more than 100 million repositories making it the largest host of source code in the world. Recent reproducibility studies have indicated that there exist good and bad practices in writing these notebooks which can affect their overall reproducibility. We present ReproduceMeGit, a visualization tool for analyzing the reproducibility of Jupyter Notebooks. This will help repository users and owners to reproduce and directly analyze and assess the reproducibility of any GitHub repository containing Jupyter Notebooks. The tool provides information on the number of notebooks that were successfully reproducible, those that resulted in exceptions, those with different results from the original notebooks, etc. Each notebook in the repository along with the provenance information of its execution can also be exported in RDF with the integration of the ProvBook tool.