Virtual Markov chains

TL;DR

This paper introduces virtual Markov chains as a projective limit of finite state Markov chains, defining virtual initial distributions and transition matrices, and explores their properties and limitations.

Contribution

It formalizes the concept of virtual Markov chains, extending classical Markov chain theory to an infinite-dimensional setting with new structural insights.

Findings

Law of VMCs characterized by VID and VTM

Failure of Birkhoff-von Neumann theorem in virtual setting

Analysis of convex sets associated with VMCs

Abstract

We introduce the space of virtual Markov chains (VMCs) as a projective limit of the spaces of all finite state space Markov chains (MCs), in the same way that the space of virtual permutations is the projective limit of the spaces of all permutations of finite sets. We introduce the notions of virtual initial distribution (VID) and a virtual transition matrix (VTM), and we show that the law of any VMC is uniquely characterized by a pair of a VID and VTM which have to satisfy a certain compatibility condition. Lastly, we study various properties of compact convex sets associated to the theory of VMCs, including that the Birkhoff-von Neumann theorem fails in the virtual setting.