Topic-oriented Adversarial Attacks against Black-box Neural Ranking Models

TL;DR

This paper introduces a novel topic-oriented adversarial attack framework against neural ranking models, using reinforcement learning to generate imperceptible perturbations that promote target documents across multiple queries, revealing security risks.

Contribution

It proposes a new attack task and a reinforcement learning-based framework for black-box, topic-oriented adversarial attacks on neural ranking models, improving attack effectiveness.

Findings

The framework significantly outperforms existing attack strategies.

It demonstrates the vulnerability of neural ranking models to topic-oriented adversarial attacks.

Potential risks for real-world deployment of NRMs are highlighted.

Abstract

Neural ranking models (NRMs) have attracted considerable attention in information retrieval. Unfortunately, NRMs may inherit the adversarial vulnerabilities of general neural networks, which might be leveraged by black-hat search engine optimization practitioners. Recently, adversarial attacks against NRMs have been explored in the paired attack setting, generating an adversarial perturbation to a target document for a specific query. In this paper, we focus on a more general type of perturbation and introduce the topic-oriented adversarial ranking attack task against NRMs, which aims to find an imperceptible perturbation that can promote a target document in ranking for a group of queries with the same topic. We define both static and dynamic settings for the task and focus on decision-based black-box attacks. We propose a novel framework to improve topic-oriented attack performance based on a surrogate ranking model. The attack problem is formalized as a Markov decision process (MDP) and addressed using reinforcement learning. Specifically, a topic-oriented reward function guides the policy to find a successful adversarial example that can be promoted in rankings to as many queries as possible in a group. Experimental results demonstrate that the proposed framework can significantly outperform existing attack strategies, and we conclude by re-iterating that there exist potential risks for applying NRMs in the real world.