Implicit score matching meets denoising score matching: improved rates of convergence and log-density Hessian estimation

TL;DR

This paper compares implicit and denoising score matching methods for estimating score functions, showing they adapt to low-dimensional structures and enable Hessian estimation without the curse of dimensionality, supporting diffusion model sampling.

Contribution

It demonstrates that implicit score matching achieves convergence rates comparable to denoising score matching and allows Hessian estimation in high dimensions, justified by new inequalities.

Findings

Implicit score matching adapts to intrinsic data dimension.

Both methods enable Hessian estimation without curse of dimensionality.

Supports convergence of ODE-based samplers for diffusion models.

Abstract

We study the problem of estimating the score function using both implicit score matching and denoising score matching. Assuming that the data distribution exhibiting a low-dimensional structure, we prove that implicit score matching is able not only to adapt to the intrinsic dimension, but also to achieve the same rates of convergence as denoising score matching in terms of the sample size. Furthermore, we demonstrate that both methods allow us to estimate log-density Hessians without the curse of dimensionality by simple differentiation. This justifies convergence of ODE-based samplers for generative diffusion models. Our approach is based on Gagliardo-Nirenberg-type inequalities relating weighted $L^2$-norms of smooth functions and their derivatives.