RecPS: Privacy Risk Scoring for Recommender Systems

TL;DR

RecPS introduces a novel privacy risk scoring method for recommender systems, enabling users and developers to quantify and understand the sensitivity of user interactions and improve privacy-aware model development.

Contribution

The paper proposes RecPS, a new privacy scoring framework based on membership inference attacks, extending differential privacy concepts to interaction and user levels in RecSys.

Findings

RecPS effectively quantifies privacy risks in RecSys data.

RecLiRA provides high-quality membership inference for privacy scoring.

Experiments demonstrate RecPS's utility in risk assessment and model unlearning.

Abstract

Recommender systems (RecSys) have become an essential component of many web applications. The core of the system is a recommendation model trained on highly sensitive user-item interaction data. While privacy-enhancing techniques are actively studied in the research community, the real-world model development still depends on minimal privacy protection, e.g., via controlled access. Users of such systems should have the right to choose \emph{not} to share highly sensitive interactions. However, there is no method allowing the user to know which interactions are more sensitive than others. Thus, quantifying the privacy risk of RecSys training data is a critical step to enabling privacy-aware RecSys model development and deployment. We propose a membership-inference attack (MIA)- based privacy scoring method, RecPS, to measure privacy risks at both the interaction and user levels. The RecPS interaction-level score definition is motivated and derived from differential privacy, which is then extended to the user-level scoring method. A critical component is the interaction-level MIA method RecLiRA, which gives high-quality membership estimation. We have conducted extensive experiments on well-known benchmark datasets and RecSys models to show the unique features and benefits of RecPS scoring in risk assessment and RecSys model unlearning.