Formal Error Bounds for the State Space Reduction of Markov Chains

TL;DR

This paper develops formal error bounds for approximating Markov chains via state space reduction, applicable to transient and stationary distributions, and compares these bounds with existing concepts like lumpability and aggregatability.

Contribution

It extends the theory of error bounds for Markov chain approximation, including both discrete and continuous-time cases, and explores algorithms for low-error state aggregation.

Findings

Derived bounds for error growth in reduced Markov chains

Compared error bounds with concepts like lumpability and aggregatability

Conducted initial experiments on aggregation algorithms

Abstract

We study the approximation of a Markov chain on a reduced state space, for both discrete- and continuous-time Markov chains. In this context, we extend the existing theory of formal error bounds for the approximated transient distributions. As a special case, we consider aggregated (or lumped) Markov chains, where the state space reduction is achieved by partitioning the state space into macro states. In the discrete-time setting, we bound the stepwise increment of the error, and in the continuous-time setting, we bound the rate at which the error grows. In addition, the same error bounds can also be applied to bound how far an approximated stationary distribution is from stationarity. Subsequently, we compare these error bounds with relevant concepts from the literature, such as exact and ordinary lumpability, as well as deflatability and aggregatability. These concepts define stricter than necessary conditions to identify settings in which the aggregation error is zero. We also consider possible algorithms for finding suitable aggregations for which the formal error bounds are low, and we analyse first experiments with these algorithms on a range of different models.