An Efficient High-Dimensional Gradient Estimator for Stochastic Differential Equations

TL;DR

This paper introduces a new unbiased gradient estimator for high-dimensional stochastic differential equations that maintains stable computation time as the dimension grows, improving efficiency in complex models.

Contribution

The paper presents the generator gradient estimator, a novel method that scales efficiently with high dimensions and is applicable to general SDEs with jumps, outperforming existing methods.

Findings

Achieves near-constant computation time as dimension increases

Outperforms pathwise differentiation in efficiency

Maintains low variance in gradient estimates

Abstract

Overparameterized stochastic differential equation (SDE) models have achieved remarkable success in various complex environments, such as PDE-constrained optimization, stochastic control and reinforcement learning, financial engineering, and neural SDEs. These models often feature system evolution coefficients that are parameterized by a high-dimensional vector $\theta \in \mathbb{R}^n$, aiming to optimize expectations of the SDE, such as a value function, through stochastic gradient ascent. Consequently, designing efficient gradient estimators for which the computational complexity scales well with $n$ is of significant interest. This paper introduces a novel unbiased stochastic gradient estimator--the generator gradient estimator--for which the computation time remains stable in $n$. In addition to establishing the validity of our methodology for general SDEs with jumps, we also perform numerical experiments that test our estimator in linear-quadratic control problems parameterized by high-dimensional neural networks. The results show a significant improvement in efficiency compared to the widely used pathwise differentiation method: Our estimator achieves near-constant computation times, increasingly outperforms its counterpart as $n$ increases, and does so without compromising estimation variance. These empirical findings highlight the potential of our proposed methodology for optimizing SDEs in contemporary applications.