PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models

TL;DR

This paper introduces POISONPROMPT, a backdoor attack method that compromises prompt-based large language models, revealing significant security vulnerabilities and emphasizing the need for improved defenses.

Contribution

The paper presents a novel backdoor attack technique specifically designed for prompt-based LLMs, demonstrating its effectiveness across multiple models, prompts, and datasets.

Findings

POISONPROMPT successfully compromises prompt-based LLMs.

The attack maintains high fidelity and robustness.

Security threats are significant and warrant further research.

Abstract

Prompts have significantly improved the performance of pretrained Large Language Models (LLMs) on various downstream tasks recently, making them increasingly indispensable for a diverse range of LLM application scenarios. However, the backdoor vulnerability, a serious security threat that can maliciously alter the victim model's normal predictions, has not been sufficiently explored for prompt-based LLMs. In this paper, we present POISONPROMPT, a novel backdoor attack capable of successfully compromising both hard and soft prompt-based LLMs. We evaluate the effectiveness, fidelity, and robustness of POISONPROMPT through extensive experiments on three popular prompt methods, using six datasets and three widely used LLMs. Our findings highlight the potential security threats posed by backdoor attacks on prompt-based LLMs and emphasize the need for further research in this area.