Reduction and classification of higher-order Markov chains

TL;DR

This paper introduces a skeleton-based framework to analyze the structure of higher-order Markov chains, enabling efficient classification of their recurrent classes and periods.

Contribution

It proposes a novel skeleton concept that captures zero-probability transition patterns, simplifying the analysis of complex higher-order Markov chains.

Findings

Skeleton structure determines recurrent classes and periods.

The approach reduces computational complexity in Markov chain analysis.

Example shows skeleton can have smaller order than the original chain.

Abstract

We study the class structure of finite-alphabet Markov chains with arbitrary memory length. To capture the structural constraints induced by prohibited transitions, we introduce the skeleton of a higher-order transition kernel, defined as a reduced set of contexts encoding all essential zero-probability patterns. To each skeleton we associate a binary transition matrix. We show that the communicating class structure of this matrix completely determines the recurrent classes of the original higher-order Markov chain, along with their periods. As a consequence, simple criteria for essential irreducibility and periodicity follow directly from the skeleton, without constructing the full first-order representation on the enlarged state space. From a practical perspective, this approach can yield significant computational gains. An example illustrates how the skeleton may have substantially smaller order than the original chain.