The Derivation of Markov Chain Properties using Generalized Matrix Inverses

TL;DR

This paper surveys how generalized matrix inverses can be used to analyze key properties of finite Markov chains, such as stationary distributions and passage times, by solving linear systems involving I - P.

Contribution

It provides a comprehensive overview of applying generalized inverses to derive Markov chain properties, highlighting their importance in solving singular linear systems.

Findings

Generalized inverses facilitate solutions to Markov chain equations.

They enable computation of stationary distributions and passage times.

The survey consolidates existing methods and applications.

Abstract

The analysis of many problems of interest associated with Markov chains, e.g. stationary distributions, moments of first passage time distributions and moments of occupation time random variables, involves the solution of a system of linear equations involving I - P, where P is the transition matrix of a finite, irreducible, discrete time Markov chain. Generalized inverses play an important role in the solution of such singular sets of equations. In this presentation we survey the application of generalized matrix inverses to the aforementioned problems.