Towards Sharing Task Environments to Support Reproducible Evaluations of Interactive Recommender Systems

TL;DR

This paper advocates for sharing comprehensive task environments in recommender systems research to enhance reproducibility, proposing a logical architecture to clarify core components and sharing requirements.

Contribution

It introduces a high-level logical architecture for RS task environments, facilitating better understanding and sharing for reproducible evaluations.

Findings

Proposes a logical architecture for RS task environments

Clarifies differences between environments, datasets, and simulations

Lays groundwork for standardized sharing practices

Abstract

Beyond sharing datasets or simulations, we believe the Recommender Systems (RS) community should share Task Environments. In this work, we propose a high-level logical architecture that will help to reason about the core components of a RS Task Environment, identify the differences between Environments, datasets and simulations; and most importantly, understand what needs to be shared about Environments to achieve reproducible experiments. The work presents itself as valuable initial groundwork, open to discussion and extensions.