SRC-Flow: Compact Semantic Representations Enable Normalizing Flows for Image Generation

TL;DR

SRC-Flow introduces a semantic representation compressor that enables normalizing flows to generate high-quality images efficiently by operating in a compact semantic space, achieving state-of-the-art results.

Contribution

The paper proposes SRC-Flow, a novel method that compresses high-dimensional features into a low-dimensional semantic space for improved flow-based image generation.

Findings

Achieves state-of-the-art gFID scores of 1.65 and 2.07 on ImageNet at 256x256 and 512x512 resolutions.

Enables exact likelihood computation and deterministic invertible sampling in the semantic space.

Outperforms previous normalizing flow methods in large-scale image generation.

Abstract

Normalizing flows (NFs) provide exact likelihoods and deterministic invertible sampling, but have historically lagged behind diffusion models for large-scale image generation. We identify a key obstacle: NFs are required to learn a single invertible transport over the full ambient space, making them highly sensitive to high-dimensional representations. This leads to a semantic-capacity mismatch in modern visual representation spaces, where semantic information is compact but encoded in overcomplete features. We propose SRC-Flow, which introduces a Semantic Representation Compressor (SRC) to compact high-dimensional RAE features into a low-dimensional semantic space before flow modeling and preserve reconstruction through the frozen RAE decoder. This compact space reduces the modeling burden of NFs and enables effective likelihood-based generation in semantic representation space. We further adopt constant noise regularization tailored to the fixed unconditional bijection learned by flows. On ImageNet $256 \times 256$ and $512 \times 512$, SRC-Flow achieves state-of-the-art generation quality among normalizing flow methods, with gFID scores of 1.65 and 2.07 under classifier-free guidance, while retaining exact likelihood computation in the compact semantic representation space and deterministic invertible sampling at the flow level. Codes and models will be available at https://github.com/longtaojiang/SRC-Flow.