Context is the Key: Backdoor Attacks for In-Context Learning with Vision Transformers

TL;DR

This paper reveals new backdoor attack methods on vision transformers that exploit in-context learning, demonstrating high success rates and showing that existing defenses are largely ineffective.

Contribution

It introduces task-specific and general backdoor attacks on vision transformers using data poisoning, highlighting vulnerabilities in in-context learning models.

Findings

Achieved up to 89.90% degradation on target tasks.

Successfully attacked all tested models with up to 13× degradation.

Prompt and fine-tuning defenses are largely ineffective.

Abstract

Due to the high cost of training, large model (LM) practitioners commonly use pretrained models downloaded from untrusted sources, which could lead to owning compromised models. In-context learning is the ability of LMs to perform multiple tasks depending on the prompt or context. This can enable new attacks, such as backdoor attacks with dynamic behavior depending on how models are prompted. In this paper, we leverage the ability of vision transformers (ViTs) to perform different tasks depending on the prompts. Then, through data poisoning, we investigate two new threats: i) task-specific backdoors where the attacker chooses a target task to attack, and only the selected task is compromised at test time under the presence of the trigger. At the same time, any other task is not affected, even if prompted with the trigger. We succeeded in attacking every tested model, achieving up to 89.90\% degradation on the target task. ii) We generalize the attack, allowing the backdoor to affect \emph{any} task, even tasks unseen during the training phase. Our attack was successful on every tested model, achieving a maximum of $13\times$ degradation. Finally, we investigate the robustness of prompts and fine-tuning as techniques for removing the backdoors from the model. We found that these methods fall short and, in the best case, reduce the degradation from 89.90\% to 73.46\%.