RecAD: Towards A Unified Library for Recommender Attack and Defense

TL;DR

RecAD introduces a comprehensive, open-source benchmarking library for evaluating attack and defense strategies in recommender systems, promoting reproducibility and credibility in this research area.

Contribution

It provides a unified, sustainable benchmarking pipeline integrating datasets, code, and evaluation metrics for recommender attack and defense research.

Findings

Establishes a standardized benchmark for fair comparison.

Facilitates reproducible and transparent research.

Supports diverse datasets and evaluation protocols.

Abstract

In recent years, recommender systems have become a ubiquitous part of our daily lives, while they suffer from a high risk of being attacked due to the growing commercial and social values. Despite significant research progress in recommender attack and defense, there is a lack of a widely-recognized benchmarking standard in the field, leading to unfair performance comparison and limited credibility of experiments. To address this, we propose RecAD, a unified library aiming at establishing an open benchmark for recommender attack and defense. RecAD takes an initial step to set up a unified benchmarking pipeline for reproducible research by integrating diverse datasets, standard source codes, hyper-parameter settings, running logs, attack knowledge, attack budget, and evaluation results. The benchmark is designed to be comprehensive and sustainable, covering both attack, defense, and evaluation tasks, enabling more researchers to easily follow and contribute to this promising field. RecAD will drive more solid and reproducible research on recommender systems attack and defense, reduce the redundant efforts of researchers, and ultimately increase the credibility and practical value of recommender attack and defense. The project is released at https://github.com/gusye1234/recad.