The Evolving Nature of Latent Spaces: From GANs to Diffusion

TL;DR

This paper explores how internal representations in generative models have evolved from GANs to diffusion models, highlighting the shift from unified to distributed latent spaces and proposing a new conceptual framework.

Contribution

It introduces a distinction between strict and broad senses of synthesis, and demonstrates how diffusion models distribute representational burden, challenging traditional latent space assumptions.

Findings

Diffusion models fragment the representational burden across layers.

Traditional latent space assumptions are challenged by diffusion architectures.

The paper proposes a new conceptual framework for understanding generative AI.

Abstract

This paper examines the evolving nature of internal representations in generative visual models, focusing on the conceptual and technical shift from GANs and VAEs to diffusion-based architectures. Drawing on Beatrice Fazi's account of synthesis as the amalgamation of distributed representations, we propose a distinction between "synthesis in a strict sense", where a compact latent space wholly determines the generative process, and "synthesis in a broad sense," which characterizes models whose representational labor is distributed across layers. Through close readings of model architectures and a targeted experimental setup that intervenes in layerwise representations, we show how diffusion models fragment the burden of representation and thereby challenge assumptions of unified internal space. By situating these findings within media theoretical frameworks and critically engaging with metaphors such as the latent space and the Platonic Representation Hypothesis, we argue for a reorientation of how generative AI is understood: not as a direct synthesis of content, but as an emergent configuration of specialized processes.