Geometric Autoencoder for Diffusion Models

TL;DR

The paper introduces Geometric Autoencoder (GAE), a novel framework that enhances latent diffusion models by improving semantic alignment, reconstruction fidelity, and latent compactness, leading to state-of-the-art high-resolution image generation.

Contribution

GAE systematically addresses key challenges in latent diffusion models through optimized semantic supervision and latent normalization, surpassing existing methods in quality and stability.

Findings

Achieves a gFID of 1.82 at 80 epochs on ImageNet-1K 256x256

Reaches a gFID of 1.31 at 800 epochs without Classifier-Free Guidance

Outperforms previous state-of-the-art methods in high-resolution image generation

Abstract

Latent diffusion models have established a new state-of-the-art in high-resolution visual generation. Integrating Vision Foundation Model priors improves generative efficiency, yet existing latent designs remain largely heuristic. These approaches often struggle to unify semantic discriminability, reconstruction fidelity, and latent compactness. In this paper, we propose Geometric Autoencoder (GAE), a principled framework that systematically addresses these challenges. By analyzing various alignment paradigms, GAE constructs an optimized low-dimensional semantic supervision target from VFMs to provide guidance for the autoencoder. Furthermore, we leverage latent normalization that replaces the restrictive KL-divergence of standard VAEs, enabling a more stable latent manifold specifically optimized for diffusion learning. To ensure robust reconstruction under high-intensity noise, GAE incorporates a dynamic noise sampling mechanism. Empirically, GAE achieves compelling performance on the ImageNet-1K $256 \times 256$ benchmark, reaching a gFID of 1.82 at only 80 epochs and 1.31 at 800 epochs without Classifier-Free Guidance, significantly surpassing existing state-of-the-art methods. Beyond generative quality, GAE establishes a superior equilibrium between compression, semantic depth and robust reconstruction stability. These results validate our design considerations, offering a promising paradigm for latent diffusion modeling. Code and models are publicly available at https://github.com/sii-research/GAE.