Simultaneous Approximation of the Score Function and Its Derivatives by Deep Neural Networks

TL;DR

This paper develops a theoretical framework for using deep neural networks to simultaneously approximate the score function and its derivatives, accommodating complex data distributions with unbounded support and low-dimensional structures, while avoiding the curse of dimensionality.

Contribution

It introduces new approximation error bounds that relax traditional bounded support assumptions and extend guarantees to derivatives of any order.

Findings

Error bounds match existing literature

Bounds are free from curse of dimensionality

Guarantees extend to derivatives of any order

Abstract

We present a theory for simultaneous approximation of the score function and its derivatives, enabling the handling of data distributions with low-dimensional structure and unbounded support. Our approximation error bounds match those in the literature while relying on assumptions that relax the usual bounded support requirement. Crucially, our bounds are free from the curse of dimensionality. Moreover, we establish approximation guarantees for derivatives of any prescribed order, extending beyond the commonly considered first-order setting.