Gracefully Filtering Backdoor Samples for Generative Large Language Models without Retraining

TL;DR

This paper introduces GraCeFul, a novel frequency space gradient clustering method that effectively filters backdoor samples in generative LLMs without retraining, significantly enhancing security against backdoor attacks.

Contribution

The paper presents a new frequency space gradient clustering technique for backdoor sample filtering in generative LLMs, outperforming existing methods without retraining models.

Findings

Achieves nearly 100% recall and F1 scores in backdoor sample detection

Reduces backdoor attack success rates to 0% across multiple datasets

Generalizes effectively to Llama-2 and Vicuna models

Abstract

Backdoor attacks remain significant security threats to generative large language models (LLMs). Since generative LLMs output sequences of high-dimensional token logits instead of low-dimensional classification logits, most existing backdoor defense methods designed for discriminative models like BERT are ineffective for generative LLMs. Inspired by the observed differences in learning behavior between backdoor and clean mapping in the frequency space, we transform gradients of each training sample, directly influencing parameter updates, into the frequency space. Our findings reveal a distinct separation between the gradients of backdoor and clean samples in the frequency space. Based on this phenomenon, we propose Gradient Clustering in the Frequency Space for Backdoor Sample Filtering (GraCeFul), which leverages sample-wise gradients in the frequency space to effectively identify backdoor samples without requiring retraining LLMs. Experimental results show that GraCeFul outperforms baselines significantly. Notably, GraCeFul exhibits remarkable computational efficiency, achieving nearly 100% recall and F1 scores in identifying backdoor samples, reducing the average success rate of various backdoor attacks to 0% with negligible drops in clean accuracy across multiple free-style question answering datasets. Additionally, GraCeFul generalizes to Llama-2 and Vicuna. The codes are publicly available at https://github.com/ZrW00/GraceFul.