A Black-Box Attack Model for Visually-Aware Recommender Systems

TL;DR

This paper introduces a black-box attack model that subtly manipulates images to unfairly promote items in visually-aware recommender systems, highlighting vulnerabilities even when visual features are only modestly influential.

Contribution

The work presents a novel black-box visual attack method that creates imperceptible image perturbations to influence recommender system rankings.

Findings

Effective attack even with modest visual feature contribution

Successful influence on item scores and rankings

Demonstrated on two datasets

Abstract

Due to the advances in deep learning, visually-aware recommender systems (RS) have recently attracted increased research interest. Such systems combine collaborative signals with images, usually represented as feature vectors outputted by pre-trained image models. Since item catalogs can be huge, recommendation service providers often rely on images that are supplied by the item providers. In this work, we show that relying on such external sources can make an RS vulnerable to attacks, where the goal of the attacker is to unfairly promote certain pushed items. Specifically, we demonstrate how a new visual attack model can effectively influence the item scores and rankings in a black-box approach, i.e., without knowing the parameters of the model. The main underlying idea is to systematically create small human-imperceptible perturbations of the pushed item image and to devise appropriate gradient approximation methods to incrementally raise the pushed item's score. Experimental evaluations on two datasets show that the novel attack model is effective even when the contribution of the visual features to the overall performance of the recommender system is modest.