Max Markov Chain

TL;DR

The paper introduces Max Markov Chain (MMC), a new, efficient representation for high-order Markov chains with sparse correlations, offering better scalability and analytical solutions compared to traditional models.

Contribution

It presents MMC as a parsimonious, expressive alternative to HMCs with an analytical solution and scalable approximate methods for large domains.

Findings

MMC has an analytical solution for parameter optimization.

MMC outperforms HMC and first-order Markov chains in synthetic experiments.

Efficient approximate solutions enable MMC to scale to large domains.

Abstract

In this paper, we introduce Max Markov Chain (MMC), a novel representation for a useful subset of High-order Markov Chains (HMCs) with sparse correlations among the states. MMC is parsimony while retaining the expressiveness of HMCs. Even though parameter optimization is generally intractable as with HMC approximate models, it has an analytical solution, better sample efficiency, and the desired spatial and computational advantages over HMCs and approximate HMCs. Simultaneously, efficient approximate solutions exist for this type of chains as we show empirically, which allow MMCs to scale to large domains where HMCs and approximate HMCs would struggle to perform. We compare MMC with HMC, first-order Markov chain, and an approximate HMC model in synthetic domains with various data types to demonstrate that MMC is a valuable alternative for modeling stochastic processes and has many potential applications.