Exploiting the Vulnerability of Large Language Models via Defense-Aware Architectural Backdoor

TL;DR

This paper introduces a novel white-box backdoor attack on large language models that embeds hidden modules within the model architecture, capable of evading fine-tuning and existing defenses, posing significant security risks.

Contribution

It proposes a new architectural backdoor method with trigger detection and noise injection modules that survive retraining and evade defenses.

Findings

The attack remains effective after fine-tuning and retraining.

The backdoor can evade probability-based defense methods like BDDR.

The method is validated on multiple datasets and model architectures.

Abstract

Deep neural networks (DNNs) have long been recognized as vulnerable to backdoor attacks. By providing poisoned training data in the fine-tuning process, the attacker can implant a backdoor into the victim model. This enables input samples meeting specific textual trigger patterns to be classified as target labels of the attacker's choice. While such black-box attacks have been well explored in both computer vision and natural language processing (NLP), backdoor attacks relying on white-box attack philosophy have hardly been thoroughly investigated. In this paper, we take the first step to introduce a new type of backdoor attack that conceals itself within the underlying model architecture. Specifically, we propose to design separate backdoor modules consisting of two functions: trigger detection and noise injection. The add-on modules of model architecture layers can detect the presence of input trigger tokens and modify layer weights using Gaussian noise to disturb the feature distribution of the baseline model. We conduct extensive experiments to evaluate our attack methods using two model architecture settings on five different large language datasets. We demonstrate that the training-free architectural backdoor on a large language model poses a genuine threat. Unlike the-state-of-art work, it can survive the rigorous fine-tuning and retraining process, as well as evade output probability-based defense methods (i.e. BDDR). All the code and data is available https://github.com/SiSL-URI/Arch_Backdoor_LLM.