MARS-Gym: A Gym framework to model, train, and evaluate Recommender Systems for Marketplaces

TL;DR

MARS-Gym is an open-source framework that enables the development, training, and evaluation of reinforcement learning agents for marketplace recommender systems, addressing data processing, model design, and multi-sided evaluation.

Contribution

It provides a comprehensive pipeline and baseline implementations to facilitate research and development of recommendation algorithms in marketplaces.

Findings

Demonstrated the use of MARS-Gym on Trivago data with metric-driven analysis.

Showcased the framework's ability to evaluate recommendation, fairness, and off-policy metrics.

Bridged the gap between academic research and real-world marketplace systems.

Abstract

Recommender Systems are especially challenging for marketplaces since they must maximize user satisfaction while maintaining the healthiness and fairness of such ecosystems. In this context, we observed a lack of resources to design, train, and evaluate agents that learn by interacting within these environments. For this matter, we propose MARS-Gym, an open-source framework to empower researchers and engineers to quickly build and evaluate Reinforcement Learning agents for recommendations in marketplaces. MARS-Gym addresses the whole development pipeline: data processing, model design and optimization, and multi-sided evaluation. We also provide the implementation of a diverse set of baseline agents, with a metrics-driven analysis of them in the Trivago marketplace dataset, to illustrate how to conduct a holistic assessment using the available metrics of recommendation, off-policy estimation, and fairness. With MARS-Gym, we expect to bridge the gap between academic research and production systems, as well as to facilitate the design of new algorithms and applications.