Backdoor Pre-trained Models Can Transfer to All

TL;DR

This paper introduces a novel backdoor attack method on pre-trained NLP models that maps trigger inputs directly to specific output representations, enabling widespread and task-agnostic malicious influence.

Contribution

It presents a new backdoor attack approach that does not rely on prior task knowledge and introduces metrics for effectiveness and stealthiness in NLP models.

Findings

The method is effective across various NLP tasks and models.

It can be implemented without prior knowledge of downstream tasks.

The attack demonstrates high success and stealthiness in experiments.

Abstract

Pre-trained general-purpose language models have been a dominating component in enabling real-world natural language processing (NLP) applications. However, a pre-trained model with backdoor can be a severe threat to the applications. Most existing backdoor attacks in NLP are conducted in the fine-tuning phase by introducing malicious triggers in the targeted class, thus relying greatly on the prior knowledge of the fine-tuning task. In this paper, we propose a new approach to map the inputs containing triggers directly to a predefined output representation of the pre-trained NLP models, e.g., a predefined output representation for the classification token in BERT, instead of a target label. It can thus introduce backdoor to a wide range of downstream tasks without any prior knowledge. Additionally, in light of the unique properties of triggers in NLP, we propose two new metrics to measure the performance of backdoor attacks in terms of both effectiveness and stealthiness. Our experiments with various types of triggers show that our method is widely applicable to different fine-tuning tasks (classification and named entity recognition) and to different models (such as BERT, XLNet, BART), which poses a severe threat. Furthermore, by collaborating with the popular online model repository Hugging Face, the threat brought by our method has been confirmed. Finally, we analyze the factors that may affect the attack performance and share insights on the causes of the success of our backdoor attack.