DPAC: Distribution-Preserving Adversarial Control for Diffusion Sampling

TL;DR

This paper introduces DPAC, a diffusion guidance method that preserves distributional fidelity by projecting adversarial gradients onto the score-defined tangent space, leading to improved sample quality and robustness in diffusion models.

Contribution

We formalize the degradation in adversarial diffusion sampling as path-KL, connect it to control energy, and develop DPAC, a guidance rule that preserves distributional fidelity by tangent projection.

Findings

DPAC achieves lower FID and path-KL on ImageNet-100.

Tangent projection cancels leading error terms, improving accuracy.

DPAC remains robust to score or metric approximation errors.

Abstract

Adversarially guided diffusion sampling often achieves the target class, but sample quality degrades as deviations between the adversarially controlled and nominal trajectories accumulate. We formalize this degradation as a path-space Kullback-Leibler divergence(path-KL) between controlled and nominal (uncontrolled) diffusion processes, thereby showing via Girsanov's theorem that it exactly equals the control energy. Building on this stochastic optimal control (SOC) view, we theoretically establish that minimizing this path-KL simultaneously tightens upper bounds on both the 2-Wasserstein distance and Fr\'echet Inception Distance (FID), revealing a principled connection between adversarial control energy and perceptual fidelity. From a variational perspective, we derive a first-order optimality condition for the control: among all directions that yield the same classification gain, the component tangent to iso-(log-)density surfaces (i.e., orthogonal to the score) minimizes path-KL, whereas the normal component directly increases distributional drift. This leads to DPAC (Distribution-Preserving Adversarial Control), a diffusion guidance rule that projects adversarial gradients onto the tangent space defined by the generative score geometry. We further show that in discrete solvers, the tangent projection cancels the O({\Delta}t) leading error term in the Wasserstein distance, achieving an O({\Delta}t^2) quality gap; moreover, it remains second-order robust to score or metric approximation. Empirical studies on ImageNet-100 validate the theoretical predictions, confirming that DPAC achieves lower FID and estimated path-KL at matched attack success rates.