GENIE: Higher-Order Denoising Diffusion Solvers

TL;DR

GENIE introduces a higher-order solver for denoising diffusion models that accelerates image synthesis by leveraging higher-order gradients and Jacobian-vector products, outperforming previous methods while preserving the true generative process.

Contribution

The paper presents a novel higher-order solver for DDMs based on truncated Taylor methods, requiring only Jacobian-vector products, and demonstrates its superior performance on image generation benchmarks.

Findings

GENIE significantly accelerates diffusion-based image synthesis.

It outperforms all previous solvers in quality and speed.

The method preserves the true generative differential equation.

Abstract

Denoising diffusion models (DDMs) have emerged as a powerful class of generative models. A forward diffusion process slowly perturbs the data, while a deep model learns to gradually denoise. Synthesis amounts to solving a differential equation (DE) defined by the learnt model. Solving the DE requires slow iterative solvers for high-quality generation. In this work, we propose Higher-Order Denoising Diffusion Solvers (GENIE): Based on truncated Taylor methods, we derive a novel higher-order solver that significantly accelerates synthesis. Our solver relies on higher-order gradients of the perturbed data distribution, that is, higher-order score functions. In practice, only Jacobian-vector products (JVPs) are required and we propose to extract them from the first-order score network via automatic differentiation. We then distill the JVPs into a separate neural network that allows us to efficiently compute the necessary higher-order terms for our novel sampler during synthesis. We only need to train a small additional head on top of the first-order score network. We validate GENIE on multiple image generation benchmarks and demonstrate that GENIE outperforms all previous solvers. Unlike recent methods that fundamentally alter the generation process in DDMs, our GENIE solves the true generative DE and still enables applications such as encoding and guided sampling. Project page and code: https://nv-tlabs.github.io/GENIE.