A classification of invariant distributions and convergence of imprecise Markov chains

TL;DR

This paper studies the structure and convergence properties of imprecise Markov chains, introducing minimal permanent classes and extremal invariant distributions to understand their long-term behavior.

Contribution

It generalizes classical Markov chain theory by defining minimal permanent classes and extremal invariant distributions for imprecise chains, providing new convergence conditions.

Findings

Identification of minimal permanent classes that contain and preserve probability mass.

Definition of extremal imprecise invariant distributions determined by upper probabilities.

Conditions established for unique convergence to extremal invariant distributions.

Abstract

We analyse the structure of imprecise Markov chains and study their convergence by means of accessibility relations. We first identify the sets of states, so-called minimal permanent classes, that are the minimal sets capable of containing and preserving the whole probability mass of the chain. These classes generalise the essential classes known from the classical theory. We then define a class of extremal imprecise invariant distributions and show that they are uniquely determined by the values of the upper probability on minimal permanent classes. Moreover, we give conditions for unique convergence to these extremal invariant distributions.