Backdoor Attacks on Dense Retrieval via Public and Unintentional Triggers

TL;DR

This paper introduces a covert backdoor attack on dense retrieval systems triggered by grammatical errors, which can retrieve malicious content with high success and minimal corpus poisoning, while remaining undetectable and preserving normal performance.

Contribution

The study reveals the vulnerability of dense retrievers to grammar-error-triggered backdoor attacks, demonstrating high attack success with minimal poisoning and robustness against defenses.

Findings

High attack success rate with only 0.048% corpus poisoning

Contrastive loss increases sensitivity to grammatical errors

Hard negative sampling exacerbates backdoor susceptibility

Abstract

Dense retrieval systems have been widely used in various NLP applications. However, their vulnerabilities to potential attacks have been underexplored. This paper investigates a novel attack scenario where the attackers aim to mislead the retrieval system into retrieving the attacker-specified contents. Those contents, injected into the retrieval corpus by attackers, can include harmful text like hate speech or spam. Unlike prior methods that rely on model weights and generate conspicuous, unnatural outputs, we propose a covert backdoor attack triggered by grammar errors. Our approach ensures that the attacked models can function normally for standard queries while covertly triggering the retrieval of the attacker's contents in response to minor linguistic mistakes. Specifically, dense retrievers are trained with contrastive loss and hard negative sampling. Surprisingly, our findings demonstrate that contrastive loss is notably sensitive to grammatical errors, and hard negative sampling can exacerbate susceptibility to backdoor attacks. Our proposed method achieves a high attack success rate with a minimal corpus poisoning rate of only 0.048\%, while preserving normal retrieval performance. This indicates that the method has negligible impact on user experience for error-free queries. Furthermore, evaluations across three real-world defense strategies reveal that the malicious passages embedded within the corpus remain highly resistant to detection and filtering, underscoring the robustness and subtlety of the proposed attack \footnote{Codes of this work are available at https://github.com/ruyue0001/Backdoor_DPR.}.