Computing expected transition events in reducible Markov chains

TL;DR

This paper derives a closed-form formula to compute the expected number of transition events in reducible Markov chains during transient and steady-state phases, aiding analysis of complex stochastic processes.

Contribution

It introduces a novel, explicit expression for expected transition events in reducible Markov chains, covering transient and steady-state behaviors.

Findings

Provides a computable formula for transient transition events

Extends analysis to steady-state and long-term behaviors

Applicable to various transition-related metrics

Abstract

We present a closed-form, computable expression for the expected number of times any transition event occurs during the transient phase of a reducible Markov chain. Examples of events include time to absorption, number of visits to a state, traversals of a particular transition, loops from a state to itself, and arrivals to a state from a particular subset of states. We give an analogous expression for time-average events, which describe the steady-state behavior of reducible chains as well as the long-term behavior of irreducible chains.