Rethinking Backdoor Detection Evaluation for Language Models

TL;DR

This paper critically evaluates the robustness of current backdoor detection methods for language models, revealing their sensitivity to training intensity and exposing limitations in existing benchmarks.

Contribution

It demonstrates that existing detection methods are highly sensitive to backdoor planting strategies and highlights the need for more robust evaluation benchmarks.

Findings

Detection success varies with training intensity of backdoors

Current benchmarks may not reflect real-world robustness

Existing methods are less effective against conservatively planted backdoors

Abstract

Backdoor attacks, in which a model behaves maliciously when given an attacker-specified trigger, pose a major security risk for practitioners who depend on publicly released language models. As a countermeasure, backdoor detection methods aim to detect whether a released model contains a backdoor. While existing backdoor detection methods have high accuracy in detecting backdoored models on standard benchmarks, it is unclear whether they can robustly identify backdoors in the wild. In this paper, we examine the robustness of backdoor detectors by manipulating different factors during backdoor planting. We find that the success of existing methods based on trigger inversion or meta classifiers highly depends on how intensely the model is trained on poisoned data. Specifically, backdoors planted with more aggressive or more conservative training are significantly more difficult to detect than the default ones. Our results highlight a lack of robustness of existing backdoor detectors and the limitations in current benchmark construction.