Transportation-Inequalities, Lyapunov Stability and Sampling for Dynamical Systems on Continuous State Space

TL;DR

This paper establishes exponential concentration inequalities for discrete-time random dynamical systems with unbounded state spaces using Lyapunov functions and transport-entropy inequalities, impacting reinforcement learning and control.

Contribution

It introduces a functional analytic framework to derive concentration inequalities and links Lyapunov stability with sampling concentration for systems with unbounded states.

Findings

Exponential concentration inequalities are derived for unbounded state space systems.

Lyapunov functions imply both stability and concentration inequalities.

Results apply to reinforcement learning and control without reversibility assumptions.

Abstract

We study the concentration phenomenon for discrete-time random dynamical systems with an unbounded state space. We develop a heuristic approach towards obtaining exponential concentration inequalities for dynamical systems using an entirely functional analytic framework. We also show that existence of exponential-type Lyapunov function, compared to the purely deterministic setting, not only implies stability but also exponential concentration inequalities for sampling from the stationary distribution, via \emph{transport-entropy inequality} (T-E). These results have significant impact in \emph{reinforcement learning} (RL) and \emph{controls}, leading to exponential concentration inequalities even for unbounded observables, while neither assuming reversibility nor exact knowledge of random dynamical system (assumptions at heart of concentration inequalities in statistical mechanics and Markov diffusion processes).