Machine Learning Pipelines: Provenance, Reproducibility and FAIR Data Principles

TL;DR

This paper addresses the reproducibility crisis in machine learning by proposing methods to enhance provenance tracking, applying FAIR data principles, and demonstrating a tool to improve reproducibility in ML workflows.

Contribution

It introduces approaches for end-to-end reproducibility in ML pipelines, emphasizing provenance, FAIR data practices, and the use of ProvBook for capturing experiment provenance.

Findings

ProvBook helps capture and compare ML experiment provenance.

Applying FAIR principles improves ML reproducibility.

Preliminary results show increased reproducibility with our approach.

Abstract

Machine learning (ML) is an increasingly important scientific tool supporting decision making and knowledge generation in numerous fields. With this, it also becomes more and more important that the results of ML experiments are reproducible. Unfortunately, that often is not the case. Rather, ML, similar to many other disciplines, faces a reproducibility crisis. In this paper, we describe our goals and initial steps in supporting the end-to-end reproducibility of ML pipelines. We investigate which factors beyond the availability of source code and datasets influence reproducibility of ML experiments. We propose ways to apply FAIR data practices to ML workflows. We present our preliminary results on the role of our tool, ProvBook, in capturing and comparing provenance of ML experiments and their reproducibility using Jupyter Notebooks.