Dimension-free Score Matching and Time Bootstrapping for Diffusion Models

TL;DR

This paper introduces nearly dimension-free sample complexity bounds for diffusion models, utilizing a single function to estimate scores across noise levels, and proposes a variance reduction technique called Bootstrapped Score Matching.

Contribution

The work establishes the first nearly dimension-free bounds for score function learning in diffusion models and introduces a practical joint estimation approach and a variance reduction method.

Findings

Nearly dimension-free sample complexity bounds achieved.

A single function approximator effectively estimates scores across noise levels.

Bootstrapped Score Matching improves accuracy at higher noise levels.

Abstract

Diffusion models generate samples by estimating the score function of the target distribution at various noise levels. The model is trained using samples drawn from the target distribution by progressively adding noise. Previous sample complexity bounds have polynomial dependence on the dimension $d$, apart from a $\log(|\mathcal{H}|)$ term, where $\mathcal{H}$ is the hypothesis class. In this work, we establish the first (nearly) dimension-free sample complexity bounds, modulo the $\log(|\mathcal{H}|)$ dependence, for learning these score functions, achieving a double exponential improvement in the dimension over prior results. A key aspect of our analysis is the use of a single function approximator to jointly estimate scores across noise levels, a practical feature that enables generalization across time steps. We introduce a martingale-based error decomposition and sharp variance bounds, enabling efficient learning from dependent data generated by Markov processes, which may be of independent interest. Building on these insights, we propose Bootstrapped Score Matching (BSM), a variance reduction technique that leverages previously learned scores to improve accuracy at higher noise levels. These results provide insights into the efficiency and effectiveness of diffusion models for generative modeling.