Lookahead Diffusion Probabilistic Models for Refining Mean Estimation

TL;DR

This paper introduces lookahead diffusion probabilistic models (LA-DPMs) that improve mean estimation in diffusion models by exploiting correlations over timesteps, leading to significant performance gains without additional training.

Contribution

The paper proposes a novel lookahead approach that refines mean estimation in diffusion models by extrapolating from consecutive timestep estimates, easily integrated without fine-tuning.

Findings

Significant FID score improvements across multiple diffusion models.

Easy integration into existing models without additional training.

Consistent performance gains demonstrated through extensive experiments.

Abstract

We propose lookahead diffusion probabilistic models (LA-DPMs) to exploit the correlation in the outputs of the deep neural networks (DNNs) over subsequent timesteps in diffusion probabilistic models (DPMs) to refine the mean estimation of the conditional Gaussian distributions in the backward process. A typical DPM first obtains an estimate of the original data sample $\boldsymbol{x}$ by feeding the most recent state $\boldsymbol{z}_i$ and index $i$ into the DNN model and then computes the mean vector of the conditional Gaussian distribution for $\boldsymbol{z}_{i-1}$. We propose to calculate a more accurate estimate for $\boldsymbol{x}$ by performing extrapolation on the two estimates of $\boldsymbol{x}$ that are obtained by feeding $(\boldsymbol{z}_{i+1},i+1)$ and $(\boldsymbol{z}_{i},i)$ into the DNN model. The extrapolation can be easily integrated into the backward process of existing DPMs by introducing an additional connection over two consecutive timesteps, and fine-tuning is not required. Extensive experiments showed that plugging in the additional connection into DDPM, DDIM, DEIS, S-PNDM, and high-order DPM-Solvers leads to a significant performance gain in terms of FID score.