Wasserstein Diffusion Tikhonov Regularization

TL;DR

This paper introduces a Wasserstein-geometry-based regularization method for training discriminative models that enhances robustness to in-class variations and adversarial perturbations, with efficient computation and improved generalization.

Contribution

It proposes a novel Tikhonov-type Wasserstein diffusion regularizer leveraging Wasserstein-2 geometry for robust and flexible discriminative model training.

Findings

Improved generalization under adversarial perturbations

Enhanced robustness to large in-class variations

Efficient regularizer computation with negligible additional cost

Abstract

We propose regularization strategies for learning discriminative models that are robust to in-class variations of the input data. We use the Wasserstein-2 geometry to capture semantically meaningful neighborhoods in the space of images, and define a corresponding input-dependent additive noise data augmentation model. Expanding and integrating the augmented loss yields an effective Tikhonov-type Wasserstein diffusion smoothness regularizer. This approach allows us to apply high levels of regularization and train functions that have low variability within classes but remain flexible across classes. We provide efficient methods for computing the regularizer at a negligible cost in comparison to training with adversarial data augmentation. Initial experiments demonstrate improvements in generalization performance under adversarial perturbations and also large in-class variations of the input data.