A Likely Geometry of Generative Models

TL;DR

This paper introduces a general geometric framework for analyzing generative models using a pseudo-metric that captures high-density regions, enabling better understanding and visualization of these models without additional training.

Contribution

The authors develop a novel, model-agnostic geometric approach based on a pseudo-metric, characterizing geodesics and shortest paths in generative models through differential equations.

Findings

Curves under the new metric traverse higher density regions.

The framework allows efficient computation of shortest paths and Fréchet means.

Quantitative results show improved density traversal over baselines.

Abstract

The geometry of generative models serves as the basis for interpolation, model inspection, and more. Unfortunately, most generative models lack a principal notion of geometry without restrictive assumptions on either the model or the data dimension. In this paper, we construct a general geometry compatible with different metrics and probability distributions to analyze generative models that do not require additional training. We consider curves analogous to geodesics constrained to a suitable data distribution aimed at targeting high-density regions learned by generative models. We formulate this as a (pseudo)-metric and prove that this corresponds to a Newtonian system on a Riemannian manifold. We show that shortest paths in our framework can be characterized by a system of ordinary differential equations, which locally corresponds to geodesics under a suitable Riemannian metric. Numerically, we derive a novel algorithm to efficiently compute shortest paths and generalized Fr\'echet means. Quantitatively, we show that curves using our metric traverse regions of higher density than baselines across a range of models and datasets.