A Forward Propagation Algorithm for Online Optimization of Nonlinear Stochastic Differential Equations

TL;DR

This paper analyzes the convergence of a forward propagation algorithm designed for online optimization of nonlinear dissipative stochastic differential equations, providing theoretical guarantees and bounds on its performance.

Contribution

It offers the first convergence analysis for the forward propagation algorithm applied to nonlinear dissipative SDEs, leveraging ergodicity and PDE bounds.

Findings

Proves convergence rate based on ergodicity properties.

Establishes bounds on stochastic fluctuations around steepest descent.

Rewrites the algorithm using PDE solutions to analyze parameter evolution.

Abstract

Optimizing over the stationary distribution of stochastic differential equations (SDEs) is computationally challenging. A new forward propagation algorithm has been recently proposed for the online optimization of SDEs. The algorithm solves an SDE, derived using forward differentiation, which provides a stochastic estimate for the gradient. The algorithm continuously updates the SDE model's parameters and the gradient estimate simultaneously. This paper studies the convergence of the forward propagation algorithm for nonlinear dissipative SDEs. We leverage the ergodicity of this class of nonlinear SDEs to characterize the convergence rate of the transition semi-group and its derivatives. Then, we prove bounds on the solution of a Poisson partial differential equation (PDE) for the expected time integral of the algorithm's stochastic fluctuations around the direction of steepest descent. We then re-write the algorithm using the PDE solution, which allows us to characterize the parameter evolution around the direction of steepest descent. Our main result is a convergence theorem for the forward propagation algorithm for nonlinear dissipative SDEs.