Ergodic Theorems for discrete Markov chains

TL;DR

This paper investigates the long-term behavior of averages in discrete Markov chains, including non-irreducible cases, using elementary methods to understand convergence of state probabilities and functions of the chain.

Contribution

It extends ergodic theorems to non-irreducible Markov chains using elementary techniques, providing new insights into their long-term averages.

Findings

Established limits of state probability averages without irreducibility

Analyzed convergence of averages of functions of the chain

Provided elementary proofs for ergodic behavior in general Markov chains

Abstract

Let $X_n$ be a discrete time Markov chain with state space $S$ (countably infinite, in general) and initial probability distribution $\mu^{(0)} = (P(X_0=i_1),P(X_0=i_2),\cdots,)$. What is the probability of choosing in random some $k \in \mathbb{N}$ with $k \leq n$ such that $X_k = j$ where $j \in S$? This probability is the average $\frac{1}{n} \sum_{k=1}^n \mu^{(k)}_j$ where $\mu^{(k)}_j = P(X_k = j)$. In this note we will study the limit of this average without assuming that the chain is irreducible, using elementary mathematical tools. Finally, we study the limit of the average $\frac{1}{n} \sum_{k=1}^n g(X_k)$ where $g$ is a given function for a Markov chain not necessarily irreducible.