Minimax Rates for Hyperbolic Hierarchical Learning

TL;DR

This paper demonstrates that hyperbolic embeddings significantly reduce sample complexity for hierarchical learning tasks compared to Euclidean embeddings, achieving optimal rates through geometric advantages.

Contribution

It proves an exponential separation in sample complexity between Euclidean and hyperbolic spaces for hierarchical data, establishing hyperbolic space as optimal for such tasks.

Findings

Hyperbolic embeddings enable $O(1)$-Lipschitz realizability for hierarchical data.

Euclidean embeddings require exponential Lipschitz constants, leading to higher sample complexity.

Matching lower bounds confirm hyperbolic space's optimality in hierarchical learning.

Abstract

We prove an exponential separation in sample complexity between Euclidean and hyperbolic representations for learning on hierarchical data under standard Lipschitz regularization. For depth-$R$ hierarchies with branching factor $m$, we first establish a geometric obstruction for Euclidean space: any bounded-radius embedding forces volumetric collapse, mapping exponentially many tree-distant points to nearby locations. This necessitates Lipschitz constants scaling as $\exp(\Omega(R))$ to realize even simple hierarchical targets, yielding exponential sample complexity under capacity control. We then show this obstruction vanishes in hyperbolic space: constant-distortion hyperbolic embeddings admit $O(1)$-Lipschitz realizability, enabling learning with $n = O(mR \log m)$ samples. A matching $\Omega(mR \log m)$ lower bound via Fano's inequality establishes that hyperbolic representations achieve the information-theoretic optimum. We also show a geometry-independent bottleneck: any rank-$k$ prediction space captures only $O(k)$ canonical hierarchical contrasts.