Fast and Scalable Analytical Diffusion

TL;DR

This paper introduces GoldDiff, a scalable, training-free analytical diffusion method that dynamically identifies a small subset of data for efficient inference, enabling large-scale generative modeling like ImageNet-1K.

Contribution

The paper presents GoldDiff, a novel framework that reduces inference complexity by dynamically selecting a data subset, overcoming the scalability limitations of traditional analytical diffusion models.

Findings

Achieves 71x speedup on AFHQ dataset.

Successfully scales analytical diffusion to ImageNet-1K.

Provides theoretical guarantees for sparse score approximation.

Abstract

Analytical diffusion models offer a mathematically transparent path to generative modeling by formulating the denoising score as an empirical-Bayes posterior mean. However, this interpretability comes at a prohibitive cost: the standard formulation necessitates a full-dataset scan at every timestep, scaling linearly with dataset size. In this work, we present the first systematic study addressing this scalability bottleneck. We challenge the prevailing assumption that the entire training data is necessary, uncovering the phenomenon of Posterior Progressive Concentration: the effective golden support of the denoising score is not static but shrinks asymptotically from the global manifold to a local neighborhood as the signal-to-noise ratio increases. Capitalizing on this, we propose Dynamic Time-Aware Golden Subset Diffusion (GoldDiff), a training-free framework that decouples inference complexity from dataset size. Instead of static retrieval, GoldDiff uses a coarse-to-fine mechanism to dynamically pinpoint the ''Golden Subset'' for inference. Theoretically, we derive rigorous bounds guaranteeing that our sparse approximation converges to the exact score. Empirically, GoldDiff achieves a $\bf 71 \times$ speedup on AFHQ while matching or achieving even better performance than full-scan baselines. Most notably, we demonstrate the first successful scaling of analytical diffusion to ImageNet-1K, unlocking a scalable, training-free paradigm for large-scale generative modeling.