Exploring the Security Threats of Retriever Backdoors in Retrieval-Augmented Code Generation

TL;DR

This paper reveals that retriever backdoors in Retrieval-Augmented Code Generation pose a serious security threat, as they can be stealthily injected and exploited to produce vulnerable code at scale, bypassing current defenses.

Contribution

The authors introduce VenomRACG, a novel stealthy attack method, and demonstrate its effectiveness in exposing practical vulnerabilities in retrieval-augmented code generation systems.

Findings

Injected code as small as 0.05% of the knowledge base can manipulate retriever results.

Backdoored retrievers can cause models to generate vulnerable code in over 40% of cases.

Current defenses are ineffective against the proposed stealthy backdoor attacks.

Abstract

Retrieval-Augmented Code Generation (RACG) is increasingly adopted to enhance Large Language Models for software development, yet its security implications remain dangerously underexplored. This paper conducts the first systematic exploration of a critical and stealthy threat: backdoor attacks targeting the retriever component, which represents a significant supply-chain vulnerability. It is infeasible to assess this threat realistically, as existing attack methods are either too ineffective to pose a real danger or are easily detected by state-of-the-art defense mechanisms spanning both latent-space analysis and token-level inspection, which achieve consistently high detection rates. To overcome this barrier and enable a realistic analysis, we first developed VenomRACG, a new class of potent and stealthy attack that serves as a vehicle for our investigation. Its design makes poisoned samples statistically indistinguishable from benign code, allowing the attack to consistently maintain low detectability across all evaluated defense mechanisms. Armed with this capability, our exploration reveals a severe vulnerability: by injecting vulnerable code equivalent to only 0.05% of the entire knowledge base size, an attacker can successfully manipulate the backdoored retriever to rank the vulnerable code in its top-5 results in 51.29% of cases. This translates to severe downstream harm, causing models like GPT-4o to generate vulnerable code in over 40% of targeted scenarios, while leaving the system's general performance intact. Our findings establish that retriever backdooring is not a theoretical concern but a practical threat to the software development ecosystem that current defenses are blind to, highlighting the urgent need for robust security measures.