Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation

TL;DR

This paper introduces a computationally efficient robust optimal transport formulation that improves deep learning applications like GANs and domain adaptation by effectively handling noisy data and outliers.

Contribution

We derive a dual form of robust OT suitable for deep learning, enabling stable training in GANs and domain adaptation with noisy datasets.

Findings

Robust OT improves GAN training on noisy datasets.

Our method enhances domain adaptation accuracy.

Sample weights reflect difficulty in generation.

Abstract

Optimal Transport (OT) distances such as Wasserstein have been used in several areas such as GANs and domain adaptation. OT, however, is very sensitive to outliers (samples with large noise) in the data since in its objective function, every sample, including outliers, is weighed similarly due to the marginal constraints. To remedy this issue, robust formulations of OT with unbalanced marginal constraints have previously been proposed. However, employing these methods in deep learning problems such as GANs and domain adaptation is challenging due to the instability of their dual optimization solvers. In this paper, we resolve these issues by deriving a computationally-efficient dual form of the robust OT optimization that is amenable to modern deep learning applications. We demonstrate the effectiveness of our formulation in two applications of GANs and domain adaptation. Our approach can train state-of-the-art GAN models on noisy datasets corrupted with outlier distributions. In particular, our optimization computes weights for training samples reflecting how difficult it is for those samples to be generated in the model. In domain adaptation, our robust OT formulation leads to improved accuracy compared to the standard adversarial adaptation methods. Our code is available at https://github.com/yogeshbalaji/robustOT.