A Simple Proof of the Monotonicity of the Invariant Distribution for a Discrete Markov Chain

TL;DR

This paper provides a straightforward proof demonstrating that the invariant distribution of a finite-state discrete Markov chain exhibits monotonicity, addressing a recent open question in the field.

Contribution

It offers a simple, accessible proof of a previously unresolved question about the monotonicity of invariant distributions in finite Markov chains.

Findings

Confirmed the monotonicity property for invariant distributions

Provided a simplified proof accessible to a broad audience

Resolved a recent open question in Markov chain theory

Abstract

This note presents a simple proof of the monotonicity of the invariant distribution of a discrete Markov chain with a finite state space. This answers a question recently raised by David Siegmund.

Equations

$$p^{\prime }(i,s_0)\ge p(i,s_0),p^{\prime }(i,j)\le p(i,j)$$

$${\pi }_1=\frac{1}{{\mu }_1}$$

$${\mu }_1=1+\sum _{k=2}^{n}p(1,k){\mu }_{k1}$$

$${\mu }_{j1}=1+\sum _{k=2}^{n}p(j,k){\mu }_{k1}\text{for}j\ge 2$$

$$\hat{p}(i,1)=p(i,1)+c_i$$

$$\hat{p}(i,2)=p(i,2)-c_i$$

$$\frac{\partial p(i,2)}{\partial p(i,1)}=\underset{c_i\to 0}{\lim }\frac{p(i,2)-c_i-p(i,2)}{c_i}=-1$$

$$\frac{\partial {\mu }_1}{\partial p(1,1)}=-{\mu }_{2,1}+\sum _{k=2}^{n}p(1,k)\frac{\partial {\mu }_{k1}}{\partial p(1,1)}$$

$$\frac{\partial {\mu }_1}{\partial p(j,1)}=\sum _{k=2}^{n}p(1,k)\frac{\partial {\mu }_{k1}}{\partial p(j,1)}$$

$$\frac{\partial {\mu }_{j1}}{\partial p(1,1)}=\sum _{k=2}^{n}p(j,k)\frac{\partial {\mu }_{k1}}{\partial p(1,1)}$$

$$\frac{\partial {\mu }_{j1}}{\partial p(j,1)}=-{\mu }_{j1}+\sum _{k=2}^{n}p(j,k)\frac{\partial {\mu }_{k1}}{\partial p(j,1)}$$

$$\begin{array}{rl}\frac{\partial {\mu }_1}{\partial p(1,1)}& =-{\mu }_{2,1}+\sum _{k_1=2}^{n}p(1,k_1)\sum _{k_2=2}^{n}p(k_1,k_2)\cdots =-{\mu }_{2,1}+0<0\end{array}$$

$$\begin{split}\frac{\partial{\mu_{1}}}{\partial{p(j,1)}}&=\sum_{k=2}^{n}p(1,k)\bigg{(}-\mu_{j1}+\sum_{k_{1}=2}^{n}p(k,k_{1})\bigg{(}-\mu_{j1}+\sum_{k_{2}=2}^{n}p(k_{1},k_{2})\cdots\bigg{)}\cdots\bigg{)}<0\end{split}$$

Full text

60J10,

Markov Chain,

1 Introduction

In the 2018 Symposium on Optimal Stopping at Rice University (in memory of Larry Shepp), David Siegmund asked whether there is a simple proof of the following result.

Theorem 1.1**.**

Let $M=(S,P)$ be a (Markov) model with a finite state space $S$, transition matrix P=\big{\{}p(i,j)\big{\}} and limit (invariant) distribution $\pi>0$. Let $M^{\prime}(S,P^{\prime})$ be a model with invariant distribution $\pi^{\prime}$ with (perturbed) matrix P^{\prime}=\big{\{}p^{\prime}(i,j)\big{\}} such that for some state $s_{0}\in S$,

[TABLE]

for all $i$ and for all $j\neq s_{0}$ and at least one $p^{\prime}(i,s_{0})>p(i,s_{0})$. Then $\pi^{\prime}(s_{0})>\pi(s_{0})$.

In recent work, Isaac M. Sonin provides an alternative proof of this using the idea of a censored Markov chain [1]. Here, these techniques are not used, and instead the result is obtained through properties of the expected first return time.

Proof.

Let the number of states be $n$. Without loss of generality set $s_{0}=1$. We state the following standard results. Let $\mu_{1}$ be the expected first return time to state $1$. Then,

[TABLE]

Let $\mu_{ij}$ be the expected first hitting time to $j$ from state $i$. Then,

[TABLE]

[TABLE]

Remark 1.2**.**

It is sufficient to show that for a matrix $P$, the invariant probability $\hat{\pi}_{1}>\pi_{1}$ for a perturbed matrix \hat{P}=\big{\{}\hat{p}(i,j)\big{\}} where

[TABLE]

and

[TABLE]

for all $i$ and for all (feasible) $c_{i}\geq 0$ with at least one $c_{i}>0$ and $\hat{p}(i,j)=p(i,j)$ for all $j\neq 1,2$.

We have for all $i$

[TABLE]

Then, from Expression 1, we have

[TABLE]

and

[TABLE]

for $j\geq 2$. From Expression 1, we have

[TABLE]

and

[TABLE]

for $j\geq 2$. We may combine Expressions 1 and 1 and iterating forward, obtain

[TABLE]

In a similar fashion, we combine Expressions 1 and 1 and iterate forward,

[TABLE]

Since $\mu_{1}$ is strictly decreasing in $p(i,1)$, $\pi_{1}$ must be strictly increasing in $p(i,1)$, and the result is shown.

∎