EMPRA: Embedding Perturbation Rank Attack against Neural Ranking Models

TL;DR

EMPRA is a novel adversarial attack method that manipulates sentence embeddings to re-rank documents in neural retrieval systems, effectively promoting target documents without perceptible changes.

Contribution

Introduces EMPRA, a new black-box adversarial attack technique that manipulates embeddings to influence neural ranking models without relying on surrogate models.

Findings

Achieves 96% success in re-ranking target documents into top 10.

Effective against various state-of-the-art neural ranking models.

Does not require surrogate models, increasing attack robustness.

Abstract

Recent research has shown that neural information retrieval techniques may be susceptible to adversarial attacks. Adversarial attacks seek to manipulate the ranking of documents, with the intention of exposing users to targeted content. In this paper, we introduce the Embedding Perturbation Rank Attack (EMPRA) method, a novel approach designed to perform adversarial attacks on black-box Neural Ranking Models (NRMs). EMPRA manipulates sentence-level embeddings, guiding them towards pertinent context related to the query while preserving semantic integrity. This process generates adversarial texts that seamlessly integrate with the original content and remain imperceptible to humans. Our extensive evaluation conducted on the widely-used MS MARCO V1 passage collection demonstrate the effectiveness of EMPRA against a wide range of state-of-the-art baselines in promoting a specific set of target documents within a given ranked results. Specifically, EMPRA successfully achieves a re-ranking of almost 96% of target documents originally ranked between 51-100 to rank within the top 10. Furthermore, EMPRA does not depend on surrogate models for adversarial text generation, enhancing its robustness against different NRMs in realistic settings.