Sample Complexity Bounds for Score-Matching: Causal Discovery and Generative Modeling

TL;DR

This paper establishes statistical sample complexity bounds for score-matching methods, demonstrating their effectiveness in causal discovery and generative modeling through neural network training and error analysis.

Contribution

It provides the first rigorous bounds on sample complexity for score-matching in causal discovery and generative modeling, connecting neural network estimation to theoretical guarantees.

Findings

Accurate score estimation is achievable with deep ReLU networks using SGD.

Bounds on causal relationship recovery depend on score estimation accuracy.

Upper bounds for score-matching in generative modeling are derived.

Abstract

This paper provides statistical sample complexity bounds for score-matching and its applications in causal discovery. We demonstrate that accurate estimation of the score function is achievable by training a standard deep ReLU neural network using stochastic gradient descent. We establish bounds on the error rate of recovering causal relationships using the score-matching-based causal discovery method of Rolland et al. [2022], assuming a sufficiently good estimation of the score function. Finally, we analyze the upper bound of score-matching estimation within the score-based generative modeling, which has been applied for causal discovery but is also of independent interest within the domain of generative models.