CausalDiff: Causality-Inspired Disentanglement via Diffusion Model for Adversarial Defense

TL;DR

CausalDiff introduces a causality-inspired diffusion model that disentangles essential label-causative factors from non-causative factors to improve neural network robustness against unseen adversarial attacks.

Contribution

The paper proposes a novel causal diffusion model with a causal information bottleneck for disentangling factors, enhancing adversarial defense beyond existing methods.

Findings

Significantly outperforms state-of-the-art defenses on unseen attacks.

Achieves over 86% robustness on CIFAR-10.

Demonstrates effectiveness across multiple datasets.

Abstract

Despite ongoing efforts to defend neural classifiers from adversarial attacks, they remain vulnerable, especially to unseen attacks. In contrast, humans are difficult to be cheated by subtle manipulations, since we make judgments only based on essential factors. Inspired by this observation, we attempt to model label generation with essential label-causative factors and incorporate label-non-causative factors to assist data generation. For an adversarial example, we aim to discriminate the perturbations as non-causative factors and make predictions only based on the label-causative factors. Concretely, we propose a casual diffusion model (CausalDiff) that adapts diffusion models for conditional data generation and disentangles the two types of casual factors by learning towards a novel casual information bottleneck objective. Empirically, CausalDiff has significantly outperformed state-of-the-art defense methods on various unseen attacks, achieving an average robustness of 86.39% (+4.01%) on CIFAR-10, 56.25% (+3.13%) on CIFAR-100, and 82.62% (+4.93%) on GTSRB (German Traffic Sign Recognition Benchmark). The code is available at https://github.com/CAS-AISafetyBasicResearchGroup/CausalDiff.