A study of imprecise Markov chains: joint lower expectations and point-wise ergodic theorems

TL;DR

This paper develops a theoretical framework for imprecise Markov chains, establishing properties of lower and upper expectations, and proving a game-theoretic strong law of large numbers leading to point-wise ergodic theorems.

Contribution

It introduces a novel game-theoretic approach to imprecise Markov chains, deriving new properties and ergodic theorems for these processes.

Findings

Established joint lower and upper expectation properties.

Derived a system of non-linear equations for transition times.

Proved a game-theoretic strong law of large numbers.

Abstract

We justify and discuss expressions for joint lower and upper expectations in imprecise probability trees, in terms of the sub- and supermartingales that can be associated with such trees. These imprecise probability trees can be seen as discrete-time stochastic processes with finite state sets and transition probabilities that are imprecise, in the sense that they are only known to belong to some convex closed set of probability measures. We derive various properties for their joint lower and upper expectations, and in particular a law of iterated expectations. We then focus on the special case of imprecise Markov chains, investigate their Markov and stationarity properties, and use these, by way of an example, to derive a system of non-linear equations for lower and upper expected transition and return times. Most importantly, we prove a game-theoretic version of the strong law of large numbers for submartingale differences in imprecise probability trees, and use this to derive point-wise ergodic theorems for imprecise Markov chains.