ROSE: Robust Selective Fine-tuning for Pre-trained Language Models

TL;DR

ROSE is a novel fine-tuning method that enhances the adversarial robustness of pre-trained language models by selectively updating parameters, leading to more stable and resilient NLP models against attacks.

Contribution

ROSE introduces first-order and second-order strategies for selective parameter updates, improving robustness without sacrificing performance.

Findings

ROSE significantly improves adversarial robustness across NLP tasks.

Ensemble of ROSE variants outperforms individual strategies.

ROSE leads to flatter, wider optima in fine-tuning solutions.

Abstract

Even though the large-scale language models have achieved excellent performances, they suffer from various adversarial attacks. A large body of defense methods has been proposed. However, they are still limited due to redundant attack search spaces and the inability to defend against various types of attacks. In this work, we present a novel fine-tuning approach called \textbf{RO}bust \textbf{SE}letive fine-tuning (\textbf{ROSE}) to address this issue. ROSE conducts selective updates when adapting pre-trained models to downstream tasks, filtering out invaluable and unrobust updates of parameters. Specifically, we propose two strategies: the first-order and second-order ROSE for selecting target robust parameters. The experimental results show that ROSE achieves significant improvements in adversarial robustness on various downstream NLP tasks, and the ensemble method even surpasses both variants above. Furthermore, ROSE can be easily incorporated into existing fine-tuning methods to improve their adversarial robustness further. The empirical analysis confirms that ROSE eliminates unrobust spurious updates during fine-tuning, leading to solutions corresponding to flatter and wider optima than the conventional method. Code is available at \url{https://github.com/jiangllan/ROSE}.