On convergence of occupational measures sets of a discrete-time stochastic control system, with applications to averaging of hybrid systems

TL;DR

This paper studies the convergence of occupational measure sets in discrete-time stochastic control systems and applies these results to approximate hybrid systems with abrupt parameter changes using differential inclusions.

Contribution

It establishes the convergence of occupational measure sets to stationary probabilities and applies this to approximate hybrid systems with differential inclusions.

Findings

Occupational measure sets converge to stationary probabilities.

The set of occupational measures is convex and compact.

Hybrid systems can be approximated by differential inclusions.

Abstract

In the first part of the paper, we consider a discrete-time stochastic control system. We show that, under certain conditions, the set of random occupational measures generated by the state-control trajectories of the system as well as the set of their mathematical expectations converge (as the time horizon tends to infinity) to a convex and compact (non-random) set, which is shown to coincide with the set of stationary probabilities of the system. In the second part, we apply the results obtained in the first part to deal with a hybrid system that evolves in continuous time and is subjected to abrupt changes of certain parameters. We show that the solutions of such a hybrid system are approximated by the solutions of a differential inclusion, the right-hand side of which is defined by the limit occupational measures set, the existence and convexity of which is established in the first part of the paper.