Identifying latent distances with Finslerian geometry

TL;DR

This paper introduces a Finslerian metric for latent space navigation in generative models, which minimizes expected length and converges to the expected Riemannian metric in high dimensions, improving theoretical grounding and practical approximation.

Contribution

It proposes a Finsler metric that explicitly minimizes expected length, compares it with the expected Riemannian metric, and proves their convergence in high dimensions.

Findings

Finsler metric explicitly minimizes expected length.

Both metrics converge at a rate of O(1/D) in high dimensions.

Expected Riemannian metric is a good approximation of the Finsler metric.

Abstract

Riemannian geometry provides us with powerful tools to explore the latent space of generative models while preserving the underlying structure of the data. The latent space can be equipped it with a Riemannian metric, pulled back from the data manifold. With this metric, we can systematically navigate the space relying on geodesics defined as the shortest curves between two points. Generative models are often stochastic, causing the data space, the Riemannian metric, and the geodesics, to be stochastic as well. Stochastic objects are at best impractical, and at worst impossible, to manipulate. A common solution is to approximate the stochastic pullback metric by its expectation. But the geodesics derived from this expected Riemannian metric do not correspond to the expected length-minimising curves. In this work, we propose another metric whose geodesics explicitly minimise the expected length of the pullback metric. We show this metric defines a Finsler metric, and we compare it with the expected Riemannian metric. In high dimensions, we prove that both metrics converge to each other at a rate of $O\left(\frac{1}{D}\right)$. This convergence implies that the established expected Riemannian metric is an accurate approximation of the theoretically more grounded Finsler metric. This provides justification for using the expected Riemannian metric for practical implementations.