Generalized Adversarial Code-Suggestions: Exploiting Contexts of LLM-based Code-Completion

TL;DR

This paper introduces a generalized framework for adversarial attacks on LLM-based code assistants, highlighting their effectiveness and the limited defenses available, raising concerns about security in AI-assisted coding.

Contribution

It proposes a novel, flexible attack formulation over prompt triggers and embedding maps, extending prior work and evaluating defenses against these sophisticated attacks.

Findings

Directional-map attacks increase stealthiness

Most defenses offer limited protection

Attacks are highly effective across various scenarios

Abstract

While convenient, relying on LLM-powered code assistants in day-to-day work gives rise to severe attacks. For instance, the assistant might introduce subtle flaws and suggest vulnerable code to the user. These adversarial code-suggestions can be introduced via data poisoning and, thus, unknowingly by the model creators. In this paper, we provide a generalized formulation of such attacks, spawning and extending related work in this domain. This formulation is defined over two components: First, a trigger pattern occurring in the prompts of a specific user group, and, second, a learnable map in embedding space from the prompt to an adversarial bait. The latter gives rise to novel and more flexible targeted attack-strategies, allowing the adversary to choose the most suitable trigger pattern for a specific user-group arbitrarily, without restrictions on the pattern's tokens. Our directional-map attacks and prompt-indexing attacks increase the stealthiness decisively. We extensively evaluate the effectiveness of these attacks and carefully investigate defensive mechanisms to explore the limits of generalized adversarial code-suggestions. We find that most defenses unfortunately offer little protection only.