Better Late Than Never: Formulating and Benchmarking Recommendation Editing

TL;DR

This paper introduces the task of recommendation editing, aiming to modify existing recommendation behaviors to remove unsuitable items without retraining, and provides a benchmark and metrics for evaluation.

Contribution

It formally defines recommendation editing, proposes evaluation metrics, and introduces a benchmark with a novel loss function for this task.

Findings

The proposed method effectively rectifies known unsuitable recommendations.

Benchmark results demonstrate the method's superiority over related approaches.

Evaluation metrics accurately measure rectification success.

Abstract

Recommendation systems play a pivotal role in suggesting items to users based on their preferences. However, in online platforms, these systems inevitably offer unsuitable recommendations due to limited model capacity, poor data quality, or evolving user interests. Enhancing user experience necessitates efficiently rectify such unsuitable recommendation behaviors. This paper introduces a novel and significant task termed recommendation editing, which focuses on modifying known and unsuitable recommendation behaviors. Specifically, this task aims to adjust the recommendation model to eliminate known unsuitable items without accessing training data or retraining the model. We formally define the problem of recommendation editing with three primary objectives: strict rectification, collaborative rectification, and concentrated rectification. Three evaluation metrics are developed to quantitatively assess the achievement of each objective. We present a straightforward yet effective benchmark for recommendation editing using novel Editing Bayesian Personalized Ranking Loss. To demonstrate the effectiveness of the proposed method, we establish a comprehensive benchmark that incorporates various methods from related fields. Codebase is available at https://github.com/cycl2018/Recommendation-Editing.