Differentiating through Stochastic Differential Equations: A Primer

TL;DR

This paper provides a comprehensive guide on numerical differentiation of stochastic differential equations, clarifying methods for both Itô and Stratonovich calculus, and analyzing the reliability of different approaches.

Contribution

It introduces a clear, organized primer on differentiating through SDEs, comparing discretize-optimize and optimize-discretize methods, and explaining their behaviors in stochastic settings.

Findings

Discretize-optimize approach is reliable for both Itô and Stratonovich SDEs.

The continuous limit of backward equations explains when methods agree or diverge.

Provides practical guidance for correctly computing gradients in stochastic systems.

Abstract

Dynamical systems are essential to model various phenomena in physics, finance, economics, and are also of current interest in machine learning. A central modeling task is investigating parameter sensitivity, whether tuning atmospheric coefficients, computing financial Greeks, or optimizing neural networks. These sensitivities are mathematically expressed as derivatives of an objective function with respect to parameters of interest and are rarely available analytically, necessitating numerical methods for approximating them. While the literature for differentiation of deterministic systems is well-covered, the treatment of stochastic systems, such as stochastic differential equations (SDEs), in most curricula is less comprehensive than the subtleties arising from the interplay of noise and discretization require. This paper provides a primer on numerical differentiation of SDEs organized as a two-tale narrative. Tale 1 demonstrates differentiating through discretized SDEs, known the discretize-optimize approach, is reliable for both It\^{o} and Stratonovich calculus. Tale 2 examines the optimize-discretize approach, investigating the continuous limit of backward equations from Tale 1 corresponding to the desired gradients. Our aim is to equip readers with a clear guide on the numerical differentiation of SDEs: computing gradients correctly in both It\^{o} and Stratonovich settings, understanding when discretize-optimize and optimize-discretize agree or diverge, and developing intuition for reasoning about stochastic differentiation beyond the cases explicitly covered.