Time Averages of Markov Processes and Applications to Two-Timescale Problems

TL;DR

This paper presents a decomposition technique for time averages of Markov processes, enabling new concentration inequalities and insights into two-timescale averaging, applicable to non-autonomous SDEs and discrete processes.

Contribution

It introduces a martingale-deterministic decomposition for time averages, with explicit martingale representation, advancing analysis of two-timescale stochastic systems.

Findings

Decomposition into martingale and deterministic parts for time averages.

Explicit martingale representation in terms of semigroup gradients.

Application to Gaussian concentration inequalities and averaging principles.

Abstract

We show a decomposition into the sum of a martingale and a deterministic quantity for time averages of the solutions to non-autonomous SDEs and for discrete-time Markov processes. In the SDE case the martingale has an explicit representation in terms of the gradient of the associated semigroup or transition operator. We show how the results can be used to obtain quenched Gaussian concentration inequalities for time averages and to provide insights into the Averaging principle for two-timescale processes.