Multi-granular Adversarial Attacks against Black-box Neural Ranking Models

TL;DR

This paper introduces a reinforcement learning-based method for generating multi-granular adversarial attacks on neural ranking models, improving attack effectiveness and imperceptibility by considering perturbations at multiple levels of granularity.

Contribution

It proposes a novel reinforcement learning framework that models multi-granular adversarial attacks as a sequential decision process, addressing the combinatorial challenge of perturbation selection.

Findings

Outperforms existing attack methods in effectiveness.

Achieves higher imperceptibility of adversarial examples.

Demonstrates robustness of neural ranking models against multi-granular attacks.

Abstract

Adversarial ranking attacks have gained increasing attention due to their success in probing vulnerabilities, and, hence, enhancing the robustness, of neural ranking models. Conventional attack methods employ perturbations at a single granularity, e.g., word or sentence level, to target documents. However, limiting perturbations to a single level of granularity may reduce the flexibility of adversarial examples, thereby diminishing the potential threat of the attack. Therefore, we focus on generating high-quality adversarial examples by incorporating multi-granular perturbations. Achieving this objective involves tackling a combinatorial explosion problem, which requires identifying an optimal combination of perturbations across all possible levels of granularity, positions, and textual pieces. To address this challenge, we transform the multi-granular adversarial attack into a sequential decision-making process, where perturbations in the next attack step build on the perturbed document in the current attack step. Since the attack process can only access the final state without direct intermediate signals, we use reinforcement learning to perform multi-granular attacks. During the reinforcement learning process, two agents work cooperatively to identify multi-granular vulnerabilities as attack targets and organize perturbation candidates into a final perturbation sequence. Experimental results show that our attack method surpasses prevailing baselines in both attack effectiveness and imperceptibility.