Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributions

TL;DR

This paper provides a theoretical analysis of score-based generative models, showing they can learn sub-Gaussian distributions efficiently with dimension-independent guarantees, supported by neural network approximation results.

Contribution

It introduces a complexity measure for distributions and proves dimension-free approximation rates for SGMs learning sub-Gaussian distributions, including mixtures of Gaussians.

Findings

SGMs can approximate sub-Gaussian distributions with dimension-independent rates

Neural network approximation of the score function is dimension-free

Theoretical guarantees extend to mixtures of Gaussians

Abstract

While score-based generative models (SGMs) have achieved remarkable success in enormous image generation tasks, their mathematical foundations are still limited. In this paper, we analyze the approximation and generalization of SGMs in learning a family of sub-Gaussian probability distributions. We introduce a notion of complexity for probability distributions in terms of their relative density with respect to the standard Gaussian measure. We prove that if the log-relative density can be locally approximated by a neural network whose parameters can be suitably bounded, then the distribution generated by empirical score matching approximates the target distribution in total variation with a dimension-independent rate. We illustrate our theory through examples, which include certain mixtures of Gaussians. An essential ingredient of our proof is to derive a dimension-free deep neural network approximation rate for the true score function associated with the forward process, which is interesting in its own right.