Sliced Inner Product Gromov-Wasserstein Distances

TL;DR

This paper introduces a sliced inner product Gromov-Wasserstein distance that improves scalability and invariance properties for aligning high-dimensional heterogeneous datasets, with applications in text clustering and language models.

Contribution

It proposes a novel sliced IGW distance with rotational invariance and provides theoretical and computational analysis for high-dimensional data alignment.

Findings

Validated the theoretical properties through numerical experiments.

Demonstrated effectiveness in heterogeneous text data clustering.

Applied to compare language model representations.

Abstract

The Gromov-Wasserstein (GW) problem provides a framework for aligning heterogeneous datasets by matching their intrinsic geometry, but its statistical and computational scaling remains an issue for high-dimensional problems. Slicing techniques offer an appealing route to scalability, but, unlike Wasserstein distances, GW problems do not generally admit closed-form solutions in one-dimension. We resolve this problem for the GW problem with inner product cost (IGW), propose a sliced IGW distance that enjoys a natural rotational invariance property, and comprehensively study its structural and computational properties. Numerical experiments validating our theory are presented, followed by applications to heterogeneous clustering of text data and language model representation comparison.