ASSET: Robust Backdoor Data Detection Across a Multiplicity of Deep Learning Paradigms

TL;DR

This paper introduces ASSET, a new backdoor detection method that effectively works across supervised, self-supervised, and transfer learning paradigms, outperforming existing methods especially against challenging clean-label attacks.

Contribution

The paper presents ASSET, a novel detection technique that actively induces model behavior differences to identify backdoor samples across various deep learning settings.

Findings

ASSET outperforms existing methods in supervised learning for diverse attacks.

It effectively detects the state-of-the-art clean-label backdoor attack.

ASSET significantly improves detection rates in SSL and TL settings.

Abstract

Backdoor data detection is traditionally studied in an end-to-end supervised learning (SL) setting. However, recent years have seen the proliferating adoption of self-supervised learning (SSL) and transfer learning (TL), due to their lesser need for labeled data. Successful backdoor attacks have also been demonstrated in these new settings. However, we lack a thorough understanding of the applicability of existing detection methods across a variety of learning settings. By evaluating 56 attack settings, we show that the performance of most existing detection methods varies significantly across different attacks and poison ratios, and all fail on the state-of-the-art clean-label attack. In addition, they either become inapplicable or suffer large performance losses when applied to SSL and TL. We propose a new detection method called Active Separation via Offset (ASSET), which actively induces different model behaviors between the backdoor and clean samples to promote their separation. We also provide procedures to adaptively select the number of suspicious points to remove. In the end-to-end SL setting, ASSET is superior to existing methods in terms of consistency of defensive performance across different attacks and robustness to changes in poison ratios; in particular, it is the only method that can detect the state-of-the-art clean-label attack. Moreover, ASSET's average detection rates are higher than the best existing methods in SSL and TL, respectively, by 69.3% and 33.2%, thus providing the first practical backdoor defense for these new DL settings. We open-source the project to drive further development and encourage engagement: https://github.com/ruoxi-jia-group/ASSET.