Imprecise Markov Semigroups and their Ergodicity

TL;DR

This paper introduces imprecise Markov semigroups to model uncertainty in continuous-time Markov processes, providing long-term bounds and analyzing ergodic behavior under various general conditions.

Contribution

It develops a framework for analyzing ergodicity of imprecise Markov semigroups across increasingly general state spaces, extending prior models to include model uncertainty.

Findings

Established uniform long-term bounds under model uncertainty.

Identified regimes where imprecision bounds collapse asymptotically.

Extended analysis to general Polish metric spaces with Lipschitz observables.

Abstract

We introduce the concept of an imprecise Markov semigroup \(\mathbf Q\). It is a tool that allows us to represent ambiguity around both the transition probabilities and the invariant measure of a continuous-time Markov process via a collection of Markov semigroups, each associated with a (possibly different) Markov process. We use techniques from topology, geometry, and probability to analyze ergodic limits under model uncertainty encoded by \(\mathbf Q\). We establish long-term bounds that are uniform in the initial state and identify regimes in which the imprecision in these bounds collapses asymptotically. Our results are proved in progressively more general settings. We first assume that \(\mathbf Q\) is compact and that the state space is Euclidean or a Riemannian manifold, working with a fixed bounded observable. We then allow the state space to be standard Borel, while keeping \(\mathbf Q\) compact and the observable fixed. Finally, we drop compactness and work on Polish metric spaces of finite diameter, where we treat arbitrary bounded Lipschitz observables. The importance of our findings for the fields of artificial intelligence and computer vision is also discussed at a high level; In particular, in the study of how the probability of an output evolves over time as we perturb the input of a convolutional autoencoder.