Data-free Backdoor Removal based on Channel Lipschitzness

TL;DR

This paper introduces a data-free, efficient method for backdoor removal in neural networks by leveraging Channel Lipschitz Constants to identify and prune infected channels, achieving state-of-the-art results.

Contribution

The paper proposes a novel data-free approach using Channel Lipschitzness to detect and prune backdoor channels in DNNs, simplifying and speeding up backdoor defense.

Findings

UCLC correlates strongly with trigger-activated channel changes

CLP method is fast, simple, and data-free

Achieves state-of-the-art backdoor removal performance

Abstract

Recent studies have shown that Deep Neural Networks (DNNs) are vulnerable to the backdoor attacks, which leads to malicious behaviors of DNNs when specific triggers are attached to the input images. It was further demonstrated that the infected DNNs possess a collection of channels, which are more sensitive to the backdoor triggers compared with normal channels. Pruning these channels was then shown to be effective in mitigating the backdoor behaviors. To locate those channels, it is natural to consider their Lipschitzness, which measures their sensitivity against worst-case perturbations on the inputs. In this work, we introduce a novel concept called Channel Lipschitz Constant (CLC), which is defined as the Lipschitz constant of the mapping from the input images to the output of each channel. Then we provide empirical evidences to show the strong correlation between an Upper bound of the CLC (UCLC) and the trigger-activated change on the channel activation. Since UCLC can be directly calculated from the weight matrices, we can detect the potential backdoor channels in a data-free manner, and do simple pruning on the infected DNN to repair the model. The proposed Channel Lipschitzness based Pruning (CLP) method is super fast, simple, data-free and robust to the choice of the pruning threshold. Extensive experiments are conducted to evaluate the efficiency and effectiveness of CLP, which achieves state-of-the-art results among the mainstream defense methods even without any data. Source codes are available at https://github.com/rkteddy/channel-Lipschitzness-based-pruning.