Latent Structure Emergence in Diffusion Models via Confidence-Based Filtering

TL;DR

This paper explores how filtering initial noise seeds by confidence scores from a pre-trained classifier reveals class-specific structure in the latent space of diffusion models, enabling conditional generation without guidance.

Contribution

It demonstrates that confidence-based filtering uncovers latent class structure in diffusion models, providing a new method for conditional sample generation.

Findings

High-confidence samples show clear class separability.

Latent structure is hidden in unfiltered noise but emerges with confidence filtering.

Confidence filtering offers an alternative to guidance methods for conditional generation.

Abstract

Diffusion models rely on a high-dimensional latent space of initial noise seeds, yet it remains unclear whether this space contains sufficient structure to predict properties of the generated samples, such as their classes. In this work, we investigate the emergence of latent structure through the lens of confidence scores assigned by a pre-trained classifier to generated samples. We show that while the latent space appears largely unstructured when considering all noise realizations, restricting attention to initial noise seeds that produce high-confidence samples reveals pronounced class separability. By comparing class predictability across noise subsets of varying confidence and examining the class separability of the latent space, we find evidence of class-relevant latent structure that becomes observable only under confidence-based filtering. As a practical implication, we discuss how confidence-based filtering enables conditional generation as an alternative to guidance-based methods.