PureEBM: Universal Poison Purification via Mid-Run Dynamics of Energy-Based Models

TL;DR

PureEBM introduces a universal purification method using Langevin sampling of Energy-Based Models to defend classifiers against various data poisoning attacks with minimal impact on model generalization.

Contribution

The paper presents a novel universal data purification technique using EBMs and Langevin sampling, improving robustness against diverse poisoning attacks.

Findings

Achieves state-of-the-art defense performance against multiple poison types.

Maintains classifier generalization with minimal feature impact.

Effective even with poisoned EBM training data.

Abstract

Data poisoning attacks pose a significant threat to the integrity of machine learning models by leading to misclassification of target distribution data by injecting adversarial examples during training. Existing state-of-the-art (SoTA) defense methods suffer from limitations, such as significantly reduced generalization performance and significant overhead during training, making them impractical or limited for real-world applications. In response to this challenge, we introduce a universal data purification method that defends naturally trained classifiers from malicious white-, gray-, and black-box image poisons by applying a universal stochastic preprocessing step $\Psi_{T}(x)$, realized by iterative Langevin sampling of a convergent Energy Based Model (EBM) initialized with an image $x.$ Mid-run dynamics of $\Psi_{T}(x)$ purify poison information with minimal impact on features important to the generalization of a classifier network. We show that EBMs remain universal purifiers, even in the presence of poisoned EBM training data, and achieve SoTA defense on leading triggered and triggerless poisons. This work is a subset of a larger framework introduced in \pgen with a more detailed focus on EBM purification and poison defense.