Tree-Wasserstein Distance for High Dimensional Data with a Latent Feature Hierarchy

TL;DR

This paper introduces a novel tree-Wasserstein distance tailored for high-dimensional data with hierarchical features, enabling efficient learning of latent hierarchies and demonstrating advantages in real-world datasets.

Contribution

The paper presents a new TWD that learns latent feature hierarchies from data using hyperbolic embeddings and a tree decoding method, improving over existing TWDs.

Findings

Provably recovers the latent feature hierarchy

Efficient and scalable computation

Outperforms existing methods in applications

Abstract

Finding meaningful distances between high-dimensional data samples is an important scientific task. To this end, we propose a new tree-Wasserstein distance (TWD) for high-dimensional data with two key aspects. First, our TWD is specifically designed for data with a latent feature hierarchy, i.e., the features lie in a hierarchical space, in contrast to the usual focus on embedding samples in hyperbolic space. Second, while the conventional use of TWD is to speed up the computation of the Wasserstein distance, we use its inherent tree as a means to learn the latent feature hierarchy. The key idea of our method is to embed the features into a multi-scale hyperbolic space using diffusion geometry and then present a new tree decoding method by establishing analogies between the hyperbolic embedding and trees. We show that our TWD computed based on data observations provably recovers the TWD defined with the latent feature hierarchy and that its computation is efficient and scalable. We showcase the usefulness of the proposed TWD in applications to word-document and single-cell RNA-sequencing datasets, demonstrating its advantages over existing TWDs and methods based on pre-trained models.