FrequencyBooster: Full-Frequency Modeling for High-Fidelity Pixel Diffusion

TL;DR

FrequencyBooster introduces a full-frequency modeling framework for pixel diffusion that enhances high-frequency detail preservation and global structure, achieving state-of-the-art image generation quality efficiently.

Contribution

The paper presents a novel high-capacity decoder and a full-frequency modeling approach that surpasses prior pixel diffusion models in fidelity and efficiency.

Findings

Achieves a state-of-the-art FID of 1.60 at 256x256 resolution in 320 epochs.

Attains an FID of 1.69 at 512x512 resolution, outperforming existing models.

Effectively balances high-frequency detail preservation with global structural integrity.

Abstract

To circumvent the inherent fidelity bottlenecks and optimization misalignment of VAE-based latent diffusion, pixel-space diffusion models have emerged as a compelling end-to-end paradigm. However, existing pixel diffusion models often struggle to balance computational efficiency with the preservation of high-frequency details. They frequently resort to patch-based compression or restricted local decoding, leading to a "spectral compromise" where high-frequency and fine-grained pixel information are suppressed. To address these challenges, we propose \textbf{FrequencyBooster}, a novel framework designed to empower pixel diffusion with full-frequency modeling capabilities without prohibitive overhead. The core of our method is a high-capacity decoder that specializes in extracting exhaustive high-frequency details and low-frequency semantics, the latter of which is derived from a Diffusion Transformer (DiT) backbone. Unlike prior works that sacrifice global context for local refinement, FrequencyBooster leverages high-dimensional feature representations to maintain global structural integrity while achieving superior pixel-level precision. Extensive experiments on ImageNet demonstrate the effectiveness of our approach: our model achieves a state-of-the-art FID of \textbf{1.60} at $256 \times 256$ resolution within only 320 epochs. Furthermore, at $512 \times 512$ resolution, FrequencyBooster attains an FID of \textbf{1.69}, significantly outperforming existing pixel-space and latent-space generative models.