Heterogeneous Wasserstein Discrepancy for Incomparable Distributions

TL;DR

This paper introduces the heterogeneous Wasserstein discrepancy (HWD), a novel divergence for comparing distributions on different metric spaces, leveraging distributional slicing and embeddings to enable efficient computation and invariance properties.

Contribution

The paper proposes HWD, extending Wasserstein distance to incomparable distributions using distributional slicing and embeddings, with theoretical analysis and practical applications.

Findings

HWD preserves rotation-invariance.

Embeddings for HWD can be efficiently learned.

HWD performs well in generative modeling and query tasks.

Abstract

Optimal Transport (OT) metrics allow for defining discrepancies between two probability measures. Wasserstein distance is for longer the celebrated OT-distance frequently-used in the literature, which seeks probability distributions to be supported on the $\textit{same}$ metric space. Because of its high computational complexity, several approximate Wasserstein distances have been proposed based on entropy regularization or on slicing, and one-dimensional Wassserstein computation. In this paper, we propose a novel extension of Wasserstein distance to compare two incomparable distributions, that hinges on the idea of $\textit{distributional slicing}$, embeddings, and on computing the closed-form Wassertein distance between the sliced distributions. We provide a theoretical analysis of this new divergence, called $\textit{heterogeneous Wasserstein discrepancy (HWD)}$, and we show that it preserves several interesting properties including rotation-invariance. We show that the embeddings involved in HWD can be efficiently learned. Finally, we provide a large set of experiments illustrating the behavior of HWD as a divergence in the context of generative modeling and in query framework.