Optimization-Free Diffusion Model -- A Perturbation Theory Approach

TL;DR

This paper introduces an optimization-free diffusion model that uses perturbation theory and eigenbasis expansion to estimate scores without neural network training or forward SDE simulation, simplifying generative modeling.

Contribution

It presents a novel, optimization-free approach to diffusion models by reformulating score estimation as a linear system based on eigenbasis expansion, avoiding iterative training.

Findings

Effective on high-dimensional Boltzmann distributions

Demonstrates competitive results on real-world datasets

Reduces computational complexity of diffusion modeling

Abstract

Diffusion models have emerged as a powerful framework in generative modeling, typically relying on optimizing neural networks to estimate the score function via forward SDE simulations. In this work, we propose an alternative method that is both optimization-free and forward SDE-free. By expanding the score function in a sparse set of eigenbasis of the backward Kolmogorov operator associated with the diffusion process, we reformulate score estimation as the solution to a linear system, avoiding iterative optimization and time-dependent sample generation. We analyze the approximation error using perturbation theory and demonstrate the effectiveness of our method on high-dimensional Boltzmann distributions and real-world datasets.