Forward sensitivity analysis for contracting stochastic systems

TL;DR

This paper develops a method for estimating gradients in contracting stochastic systems, extending forward sensitivity analysis to stochastic dynamics with applications to neural networks.

Contribution

It introduces conditions for differentiability of stationary costs and generalizes forward sensitivity analysis to stochastic systems with contraction properties.

Findings

Derived an estimator for derivatives in stochastic systems

Established conditions for differentiability of stationary costs

Applied method to neural network models

Abstract

In this work we investigate gradient estimation for a class of contracting stochastic systems on a continuous state space. We find conditions on the one-step transitions, namely differentiability and contraction in a Wasserstein distance, that guarantee differentiability of stationary costs. Then we show how to estimate the derivatives, deriving an estimator that can be seen as a generalization of the forward sensitivity analysis method used in deterministic systems. We apply the results to examples, including a neural network model.