Markov Chain Decomposition Based On Total Expectation Theorem

TL;DR

This paper introduces a flexible, linear decomposition method for continuous-time Markov chains based on the total expectation theorem, simplifying analysis and enabling solutions for complex models.

Contribution

It proposes a novel, versatile decomposition approach that allows simple linear aggregation of subchain properties without complex normalization constraints.

Findings

Successfully applied to congestion-based staffing queue

Analytically solved Markov-modulated Mt/Mt/1 queue

Numerical studies demonstrate method's effectiveness on large MCs

Abstract

A divide-and-conquer approach to analyzing Markov chains (MCs) is not utilized as widely as it could be, despite its potential benefits. One primary reason for this is the fact that most MC decomposition approaches involve a complex and inflexible methodology: decomposed subchains must be disjoint, transition rates of these decomposed subchains must be altered in a way tailored to the particular MC model, and the procedure to aggregate suchains needs to incorporate a nonlinear normalization constraint, complicating the analytical expression of performance measures. In contrast, we propose a versatile yet simple decomposition method for continuous time MCs based on the total expectation theorem. Leveraging the properties of this theorem, our method has great flexibility in the choice of subchains, and the procedure to obtain expected values of interest is simply a linear summation of subchains' properties, which is not affected by the normalization constraint. We prove that to maintain the correct distribution of decomposed subchains one may use our novel termination scheme, a modification of transition rates, that ensures partial flow conservation at boundary states. This termination scheme is applicable to MCs with any structure, since the scheme depends only on the boundary-state distribution, not on the structure of the MCs. To demonstrate the generality and capability of our method, we analytically solve various models, such as a congestion-based staffing queue and a Markov-modulated Mt/Mt/1 queue. As not all systems admit an analytical solution, we complement this analysis with numerical studies of MCs with various sizes using the algorithm based on our method.