Towards A Critical Evaluation of Robustness for Deep Learning Backdoor Countermeasures

TL;DR

This paper critically evaluates the robustness of existing deep learning backdoor countermeasures, revealing their vulnerabilities and non-robust cases through formal proofs and empirical analysis, emphasizing the need for thorough robustness assessment.

Contribution

It provides the first systematic analysis of the robustness of three influential backdoor detection methods, uncovering their non-robust scenarios independent of adaptive attacks.

Findings

Neural Cleanse, ABS, and MNTD have inherent non-robust cases.

Varying tasks, models, datasets, and hyperparameters can bypass these defenses.

Recent MNTD is particularly vulnerable in certain non-robust scenarios.

Abstract

Since Deep Learning (DL) backdoor attacks have been revealed as one of the most insidious adversarial attacks, a number of countermeasures have been developed with certain assumptions defined in their respective threat models. However, the robustness of these countermeasures is inadvertently ignored, which can introduce severe consequences, e.g., a countermeasure can be misused and result in a false implication of backdoor detection. For the first time, we critically examine the robustness of existing backdoor countermeasures with an initial focus on three influential model-inspection ones that are Neural Cleanse (S&P'19), ABS (CCS'19), and MNTD (S&P'21). Although the three countermeasures claim that they work well under their respective threat models, they have inherent unexplored non-robust cases depending on factors such as given tasks, model architectures, datasets, and defense hyper-parameter, which are \textit{not even rooted from delicate adaptive attacks}. We demonstrate how to trivially bypass them aligned with their respective threat models by simply varying aforementioned factors. Particularly, for each defense, formal proofs or empirical studies are used to reveal its two non-robust cases where it is not as robust as it claims or expects, especially the recent MNTD. This work highlights the necessity of thoroughly evaluating the robustness of backdoor countermeasures to avoid their misleading security implications in unknown non-robust cases.