Adversarial Unlearning of Backdoors via Implicit Hypergradient

TL;DR

This paper introduces I-BAU, a novel implicit hypergradient-based algorithm for effectively removing backdoors from poisoned models, demonstrating superior robustness and efficiency over existing defenses across diverse attack scenarios.

Contribution

The paper presents the I-BAU algorithm utilizing implicit hypergradients for backdoor unlearning, with theoretical convergence analysis and extensive empirical validation showing improved performance and speed.

Findings

I-BAU outperforms six state-of-the-art defenses across multiple datasets and attack settings.

I-BAU is significantly faster, over 13 times quicker than the most efficient baseline.

I-BAU remains effective even with only 100 clean samples available.

Abstract

We propose a minimax formulation for removing backdoors from a given poisoned model based on a small set of clean data. This formulation encompasses much of prior work on backdoor removal. We propose the Implicit Bacdoor Adversarial Unlearning (I-BAU) algorithm to solve the minimax. Unlike previous work, which breaks down the minimax into separate inner and outer problems, our algorithm utilizes the implicit hypergradient to account for the interdependence between inner and outer optimization. We theoretically analyze its convergence and the generalizability of the robustness gained by solving minimax on clean data to unseen test data. In our evaluation, we compare I-BAU with six state-of-art backdoor defenses on seven backdoor attacks over two datasets and various attack settings, including the common setting where the attacker targets one class as well as important but underexplored settings where multiple classes are targeted. I-BAU's performance is comparable to and most often significantly better than the best baseline. Particularly, its performance is more robust to the variation on triggers, attack settings, poison ratio, and clean data size. Moreover, I-BAU requires less computation to take effect; particularly, it is more than $13\times$ faster than the most efficient baseline in the single-target attack setting. Furthermore, it can remain effective in the extreme case where the defender can only access 100 clean samples -- a setting where all the baselines fail to produce acceptable results.