Unbalanced minibatch Optimal Transport; applications to Domain Adaptation

TL;DR

This paper explores the use of unbalanced optimal transport with minibatches to improve large-scale distribution comparison, demonstrating its advantages in domain adaptation tasks through theoretical analysis and empirical results.

Contribution

It introduces unbalanced optimal transport for minibatch estimation, providing theoretical insights and showing improved performance in domain adaptation.

Findings

Unbalanced optimal transport offers more robust estimates than standard methods.

The approach achieves state-of-the-art results in domain adaptation.

Theoretical properties include unbiased estimators and concentration bounds.

Abstract

Optimal transport distances have found many applications in machine learning for their capacity to compare non-parametric probability distributions. Yet their algorithmic complexity generally prevents their direct use on large scale datasets. Among the possible strategies to alleviate this issue, practitioners can rely on computing estimates of these distances over subsets of data, {\em i.e.} minibatches. While computationally appealing, we highlight in this paper some limits of this strategy, arguing it can lead to undesirable smoothing effects. As an alternative, we suggest that the same minibatch strategy coupled with unbalanced optimal transport can yield more robust behavior. We discuss the associated theoretical properties, such as unbiased estimators, existence of gradients and concentration bounds. Our experimental study shows that in challenging problems associated to domain adaptation, the use of unbalanced optimal transport leads to significantly better results, competing with or surpassing recent baselines.