Artifact Sharing for Information Retrieval Research

TL;DR

This paper presents a new, flexible method for sharing various artifacts in Information Retrieval research, enhancing reproducibility, discoverability, and reuse beyond existing code and model sharing practices.

Contribution

It introduces an interoperable sharing framework for diverse IR artifacts, addressing the lack of standardized sharing methods for indexes and other resources.

Findings

Improved artifact discoverability and reuse in IR research.

Enhanced reproducibility through standardized sharing.

Facilitated collaboration among IR researchers.

Abstract

Sharing artifacts -- such as trained models, pre-built indexes, and the code to use them -- aids in reproducibility efforts by allowing researchers to validate intermediate steps and improves the sustainability of research by allowing multiple groups to build off one another's prior computational work. Although there are de facto consensuses on how to share research code (through a git repository linked to from publications) and trained models (via HuggingFace Hub), there is no consensus for other types of artifacts, such as built indexes. Given the practical utility of using shared indexes, researchers have resorted to self-hosting these resources or performing ad hoc file transfers upon request, ultimately limiting the artifacts' discoverability and reuse. This demonstration introduces a flexible and interoperable way to share artifacts for Information Retrieval research, improving both their accessibility and usability.