Pulling back information geometry

TL;DR

This paper introduces a method to define meaningful latent space geometries for deep generative models with complex decoders by using the Fisher-Rao metric, extending the applicability beyond Gaussian assumptions.

Contribution

It proposes using the Fisher-Rao metric to pull back the geometry to the latent space, enabling analysis of models with non-Gaussian decoders.

Findings

Enables latent geometry analysis for diverse decoder distributions.

Extends the theoretical framework beyond Gaussian decoders.

Facilitates 'black box' latent geometry exploration.

Abstract

Latent space geometry has shown itself to provide a rich and rigorous framework for interacting with the latent variables of deep generative models. The existing theory, however, relies on the decoder being a Gaussian distribution as its simple reparametrization allows us to interpret the generating process as a random projection of a deterministic manifold. Consequently, this approach breaks down when applied to decoders that are not as easily reparametrized. We here propose to use the Fisher-Rao metric associated with the space of decoder distributions as a reference metric, which we pull back to the latent space. We show that we can achieve meaningful latent geometries for a wide range of decoder distributions for which the previous theory was not applicable, opening the door to `black box' latent geometries.