# Latent Geometry and Memorization in Generative Models

## TL;DR

This paper investigates the geometry of latent spaces in generative models to distinguish between memorization and genuine learning, proposing methods to analyze output densities directly for better understanding.

## Contribution

It introduces a geometric framework for analyzing output densities in generative models, enabling clearer differentiation between memorized and novel outputs.

## Key findings

- Memorization results in delta-function densities concentrated on specific examples.
- Understanding latent geometry is essential for accurately measuring memorization.
- Proposed techniques allow direct inspection of output densities in generative models.

## Abstract

It can be difficult to tell whether a trained generative model has learned to generate novel examples or has simply memorized a specific set of outputs. In published work, it is common to attempt to address this visually, for example by displaying a generated example and its nearest neighbor(s) in the training set (in, for example, the L2 metric). As any generative model induces a probability density on its output domain, we propose studying this density directly. We first study the geometry of the latent representation and generator, relate this to the output density, and then develop techniques to compute and inspect the output density. As an application, we demonstrate that "memorization" tends to a density made of delta functions concentrated on the memorized examples. We note that without first understanding the geometry, the measurement would be essentially impossible to make.