repro_eval: A Python Interface to Reproducibility Measures of System-oriented IR Experiments

TL;DR

repro_eval is a Python tool designed to facilitate reproducibility assessments in system-oriented IR experiments, promoting standardized practices and extensibility for researchers.

Contribution

the paper introduces repro_eval, a Python package that provides measures for reproducibility in IR experiments, encouraging consistent and extensible reproducibility studies.

Findings

enables reactive reproducibility assessments

offers an extensible interface for IR reproducibility measures

aims to standardize reproducibility practices in IR research

Abstract

In this work we introduce repro_eval - a tool for reactive reproducibility studies of system-oriented information retrieval (IR) experiments. The corresponding Python package provides IR researchers with measures for different levels of reproduction when evaluating their systems' outputs. By offering an easily extensible interface, we hope to stimulate common practices when conducting a reproducibility study of system-oriented IR experiments.