RGRecSys: A Toolkit for Robustness Evaluation of Recommender Systems

TL;DR

RGRecSys is a comprehensive toolkit designed to evaluate the robustness of recommender systems across multiple dimensions, addressing a gap in existing software for holistic robustness assessment.

Contribution

The paper introduces RGRecSys, a novel software library enabling unified robustness evaluation of recommender systems under various challenging scenarios.

Findings

Provides a unified framework for robustness testing

Enables evaluation across multiple robustness dimensions

Fills a gap in existing recommender system tools

Abstract

Robust machine learning is an increasingly important topic that focuses on developing models resilient to various forms of imperfect data. Due to the pervasiveness of recommender systems in online technologies, researchers have carried out several robustness studies focusing on data sparsity and profile injection attacks. Instead, we propose a more holistic view of robustness for recommender systems that encompasses multiple dimensions - robustness with respect to sub-populations, transformations, distributional disparity, attack, and data sparsity. While there are several libraries that allow users to compare different recommender system models, there is no software library for comprehensive robustness evaluation of recommender system models under different scenarios. As our main contribution, we present a robustness evaluation toolkit, Robustness Gym for RecSys (RGRecSys -- https://www.github.com/salesforce/RGRecSys), that allows us to quickly and uniformly evaluate the robustness of recommender system models.