Finding the Smallest Possible Exact Aggregation of a Markov Chain

TL;DR

This paper introduces an Arnoldi iteration-based algorithm to find the smallest exact aggregation of a Markov chain, surpassing existing methods in certain cases and highlighting the need for runtime improvements.

Contribution

It presents a novel Arnoldi iteration algorithm for minimal exact Markov chain aggregation, extending beyond traditional approximation techniques.

Findings

The Arnoldi-based method can find exact aggregations where others fail.

Exact minimal aggregations are achievable with the proposed algorithm.

Runtime remains a challenge for practical applications.

Abstract

Markov chains are an important tool for modelling and evaluating systems in computer science, economics, biology and numerous other fields. Thus, approximating Markov chains is a useful tool for decreasing the computational effort needed for analysis by reducing the size of the state space. So far, most approximative algorithms focused on finding approximations, which are, again, Markov chains. We remove this restriction and present an algorithm using the Arnoldi iteration. Further, we show that if these approximations are without error, they are of minimal size. Lastly, we implement the algorithm for Markov chains via the Arnoldi iteration. If another type of aggregation, so-called Exlump aggregations, can find an exact aggregation for a Markov chain, they usually yield similar or smaller errors at much lower runtime. On the contrary, approximations computed via the Arnoldi iteration can find usable aggregations where Exlump yields none. Still, runtime is a primary concern, as our criterion for determining the approximate state space size is exceptionally costly. As such, further work is needed to decrease the runtime of the Arnoldi approach.