On the final limit of a transition matrix

TL;DR

This paper introduces an efficient algorithm to compute the limit of the matrix exponential exp(tA) as t approaches infinity, where A is a finite-dimensional intensity matrix, aiding long-term behavior analysis.

Contribution

The paper presents a novel efficient algorithm specifically designed for calculating the limit of matrix exponentials of intensity matrices as time tends to infinity.

Findings

Algorithm successfully computes limits for finite-dimensional intensity matrices.

Method improves computational efficiency over existing approaches.

Applicable to long-term analysis in stochastic processes.

Abstract

An efficient algorithm is presented for computation of the limit of exp(t A) for t towards infinity where A denotes an intensity matrix of finite dimension.

Equations

$$\exp (tB)=\sum _{n=0}^{\infty }\frac{t^n}{n!}{B}^n,t\in \mathbb{R}.$$

$$Y_{\infty }{Y}_{\infty }=Y_{\infty }\text{ and }\underset{t\to \infty }{\lim }{Y}^{\prime }(t)=Y_{\infty }A=O.$$

$$y(t)=\underset{s\to \infty }{\lim }{y}_{0}Y(s)Y(t)=\underset{s\to \infty }{\lim }{y}_{0}Y(s+t)=y_{\infty }$$

$$\sum _{n=0}^N\frac{t^n}{n!}{B}^n=E+tB+...+\frac{t^N}{N!}{B}^N,N\in \mathbb{N},t\in \mathbb{R}$$

$$\sum _{j\in Z}{q}_{ij}=\frac{1}{d_i}\sum _{j\ne i}{b}_{ij}=\frac{-b_{ii}}{d_i}=1.$$

$$\text{diag}(z{I}_1-B_1,...,z{I}_k-B_k{)}^{-1}=\text{diag}((z{I}_1-B_1{)}^{-1},...,(z{I}_k-B_k{)}^{-1})$$

$$(zI-B{)}^{-1}={\left(\begin{array}{cc}A(z)& O\\ B_{T,R}& C(z)\end{array}\right)}^{-1}=\left(\begin{array}{cc}A(z{)}^{-1}& O\\ -C(z{)}^{-1}{B}_{T,R}A(z{)}^{-1}& C(z{)}^{-1}\end{array}\right).$$

$$P_{T,R}=-B_T^{-1}{B}_{T,R}{P}_R.$$

$$Y^{\prime }(t{)}_{T,R}=(Y(t)B{)}_{T,R}=B_{T,R}+B_{T,R}\sum _{n=1}^{\infty }\frac{t^n}{n!}{B}_R^n+B_T\sum _{n=1}^{\infty }\frac{t^n}{n!}{B^n}_{T,R}$$

$$Y^{\prime }(t{)}_{T,R}=B_{T,R}Y(t{)}_R+B_{T}Y(t{)}_{T,R}.$$

$$B_T{P}_{T,\{j\}}=-B_{T,R}{P}_{R,\{j\}}=-p_{jj}\sum _{j\in J}{B}_{T,\{j\}}=p_{jj}{B}_T(f_{i,J}{)}_{i\in T}.$$

$$(c_{ij}{)}_{i,j\in Z}:=A+A^2+...+A^{n-1}.$$

Full text

On the final limit of a transition matrix

Helmut Kahl

Munich University of Applied Sciences

Lothstr. 34

D-80335 München

Germany

kahl@hm.edu

Abstract. For a finite intensity matrix $B$ the final limit of its transition matrix $\exp(tB)$ exists. This is a well-known fact in the realm of continous-time Markov processes where it is proven by probability theoretic means. A simple proof is presented with help of a Tauberian theorem of complex analytic functions which is used also in [2] to proof the prime number theorem. Furthermore the final limit is computed.

Mathematical Subject Cassification(2010). 15B51, 30D15, 60J27.

Keywords. intensity/stochastic matrix, Laplace transform, Tauberian theory.

1 Introduction

Square matrices with finite index set $Z$ are considered. A right intensity matrix $B=(b_{ij})$ is defined by $b_{ij}\geq 0$ for all $i,j\in Z$ with $i\neq j$ and by vanishing row sums $\sum\limits_{j\in Z}b_{ij}=0$ for all $i\in Z$. Its transition matrix is111The probability distribution P of the corresponding Markov process $\{X_{t}\}_{t\geq 0}$ on the state space $Z$ with initial probability row vector $\pi$ fulfills $(\textnormal{P}\{X_{t}=j\})_{j\in Z}=\pi\exp(tB),t\geq 0$. By matrix transposition everything may be formulated also with column vectrors (and vanishing column sums of $B$).

[TABLE]

Theorem 1**.**

For a right intensity matrix $B$ the limit $\lim\limits_{t\to\infty}\exp(tB)$ exists.

Proof.

A probability theoretic proof is given in [6], thm. 5.4.6. The assertion follows also by the Tauberian theorem [1], ch. III, thm. 7.1 from the complex analytic Proposition 5 deferred to the next section. Another more elementary proof is given by [4]: The limit existence follows from the boundedness of $\exp(tB)$ (s. Lemma 3) and from the fact that the non-zero eigenvalues of $B$ have negative real part (s. proof of Proposition 5) by Jordan decomposition of $B$. ∎

2 Complex analytic proof of the theorem

First we recall some theory of homogenous linear differential equation systems with constant coefficients. The zero matrix is denoted by $O$, the identity matrix by $I$.

Lemma 2**.**

For a complex square matrix $A$ the matrix $Y(z):=\exp(zA)$ fulfills $Y(0)=I$, $Y^{\prime}=YA=AY$ and $Y(w+z)=Y(w)+Y(z)$ for all $w,z\in\mathbb{C}$. Let $y_{0}$ be a complex row vector s.t. $y_{\infty}:=\lim\limits_{t\to\infty}y_{0}Y(t)$ exists. Then it holds $y_{\infty}A=0$. If $Y_{\infty}:=\lim\limits_{t\to\infty}Y(t)$ exists it holds

[TABLE]

If $Y$ is bounded on the positve real axis every entry of the Laplace transform $z\mapsto(zI-A)^{-1}$ of $Y$ is holomorphic on the right half plane $\Re(z)>0$.

Proof.

The formulas for $Y$ follow easily from the definition. For proof of the formulas for $y_{\infty}$ observe that the function $y(t):=y_{\infty}Y(t)$ fulfills

[TABLE]

for all $t\in\mathbb{R}$. So we have indeed $y_{\infty}A=y(t)A=y^{\prime}(t)=0$. In case $Y_{\infty}$ exists it holds $Y(t)Y_{\infty}=Y_{\infty}$ for all $t\in\mathbb{R}$. Therefrom $Y_{\infty}Y_{\infty}=Y_{\infty}$ follows. By choosing canonical unit vectors for $y_{0}$ we see $Y_{\infty}A=O$. Laplace transformation of $Y^{\prime}=YA$ in each entry yields $z\hat{Y}(z)-Y(0)=\hat{Y}(z)A$ for the Laplace transform $\hat{Y}$ of $Y$. Because of $Y(0)=I$ we have indeed $\hat{Y}(z)=(zI-A)^{-1}$. Boundedness of $Y$ implies boundedness of each entry of $Y$. So $\hat{Y}$ is well defined on $\Re(z)>0$. Holomorphy follows from $M^{-1}=\textnormal{adj}M/|M|$ (s. [5], prop. 3.3) with the adjugate $\textnormal{adj}M$ of a matrix $M$ with determinant $|M|\neq 0$. ∎

A square matrix $P=(p_{ij})$ is called (right) stochastic when all its entries $p_{ij}\geq 0$ are non-negative and all its row sums are equal to one. Thus the row sum norm of $P$ equals one.

Lemma 3**.**

For a right intensity matrix $B$ its transition matrix $\exp(tB)$ is stochastic, hence a bounded function in $t\in\mathbb{R}$.

Proof.

The product $AB$ of any matrix $A$ with $B$ has vanishing row sums. Therefore $B^{n}$ has vanishing row sums for all $n\in\mathbb{N}$. Hence the partial sums

[TABLE]

have row sums one. By the limiting process with $N\to\infty$ the same assertion holds for $\exp(tB)$. For proving non-negativity we choose a real number $\mu$ such that $T:=B+\mu I$ has non-negative entries (like in the proof of [3], thm. 2.7). Since $\exp(tT)$ has non-negative entries and $e^{-\mu t}>0$ the assertion follows by $\exp(tB)=\exp(-\mu tI)\exp(tT)=e^{-\mu t}\exp(tT)$ (s. Lemma 2!). ∎

A square matrix $(a_{ij})$ is called reducible when there is some non-empty, proper subset $I$ of the index set $Z$ such that for all $i\in I$ and all $j\in Z\setminus I$ it holds $a_{ij}=0$. Otherwise the matrix is called irreducible. An index $r\in Z$ is called recurrent with respect to $A=(a_{ij}$) when there is some $r$ containing subset $J$ of $Z$ which for the submatrix $A_{J}:=(a_{ij})_{i,j\in J}$ is irreducible. Then $J$ is called the recurrence class of $r$. A non-recurrent index is called transient. We use also the notation $A_{I,J}:=(a_{ij})_{i\in I,j\in J}$ for subsets $I,J$ of $Z$. So we have $A_{J,J}=A_{J}$.

Lemma 4**.**

For a right intensity matrix $B$ with index set $Z$ let $D$ be the diagonal matrix with entries $d_{i}:=-b_{ii},i\in Z$. In case $d_{i}=0$ substitute $d_{i}$ by one, so that $D$ becomes invertible. For $i\in Z$ let $q_{ii}:=0$ in case $b_{ii}\neq 0$ and $q_{ii}:=1$ otherwise. For $i,j\in Z$ with $i\neq j$ let $q_{ij}:=b_{ij}/d_{i}$. This defines a stochastic matrix $Q=(q_{ij})_{i,j\in Z}$ with $B=DQ-D$. The function $x\mapsto xD$ is an isomorphism from the left null space of $B$ onto the left eigenspace of $Q$ with eigenvalue one. For every non-empty subset $J$ of $Z$ the same assertions hold for $B_{J},D_{J},Q_{J}$ instead of $B,D,Q$. The submatrix $B_{J}$ is irreducible if and only if $J$ is a recurrence class with respect to $B$. If the set $T$ of transient indices with respect to $B$ is not empty then $Z\setminus T$ is not empty and $B_{T}$ is invertible. In case $B$ is irreducible its left null space is spanned by a vector with positive coordinates.

Proof.

In case $b_{ii}\neq 0$ it holds

[TABLE]

In case $b_{ii}=0$ this row sum is also one. So $Q$ is stochastic.222For the Markov process described in the footnote above $Q$ is the transition matrix of the embedded Markov chain. The equation $B=DQ-D$ and therefrom the equation $B_{J}=D_{J}Q_{J}-D_{J},J\subset Z$ is verified readily. Hence for a row vector $x$ with index set $J$ the equation $xB_{J}=0$ is equivalent with $(xD_{J})Q_{J}=xD_{J}$. So the function $x\mapsto xD_{J}$ is an isomorphism from the left null space of $B_{J}$ onto the left eigenspace of $Q_{J}$ with eigenvalue one. The assertion about recurrence classes follows from the definition. From finiteness of $Z$ follows the existence of a recurrent index. Now assume $T\neq\emptyset$. The definition of transience implies the existence of some $i\in T$ and some $j\in Z\setminus T$ s.t. $b_{ij}\neq 0$, therefore also $q_{ij}\neq 0$. Hence $Q_{T}$ is not stochastic. According to [5], ch. 8.3.2, lem. 12 the submatrix $Q_{T}$ does not have eigenvalue one. So by the isomorphism above with $J=T$ the left null space of $B_{T}$ is zero. Thus $B_{T}$ is invertible. For proving the last assertion we may assume without loss of generality that $Z$ has more than one element. Assume $b_{ii}=0$ for some $i\in Z$. Then we have $b_{ij}=0$ for all $j\neq i$, so that $B$ is reducible. So for irreducible $B$ we have $q_{ij}=b_{ij}/d_{i}$ for all $i,j\in Z$ with $i\neq j$. Hence $Q$ is irreducible, too. By the Perron-Frobenius Theorem [5], thm. 8.2 the left eigenspace of $Q$ with eigenvalue one is spanned by a vector $y$ with positive coordinates. Since $D^{-1}$ has the positive diagonal entries $1/d_{i}$ the vector $x:=yD^{-1}$ has also positive coordinates. By the above described isomorphism $x\mapsto xD$ the left null space of $B$ is one-dimensional. ∎

Proposition 5**.**

For a right intensity matrix $B$ it exists a positive $\delta$ s.t. every holomorphic entry of the matrix valued function $z\mapsto z(zI-B)^{-1},\Re(z)>0$ is analytically continuable to the half plane $\Re(z)>-\delta$.

Proof.

By Lemma 2 & 3 the given function $F(z):=z(zI-B)^{-1}$ is indeed well-defined and entry-wise holomorphic on the half plane $\Re(z)>0$. The polynomial $z\mapsto|zI-B|$ of $B$ has finitely many roots. So $F$ is meromorphic on the whole complex plane. According to [5], prop. 5.12 all eigenvalues of $B$ are elements of the compact disc of radius $\rho:=\max\{-b_{ii}\mid i\in Z\}$ around $-\rho$. Hence the non-zero eigenvalues of $B=(b_{ij})_{i,j\in Z}$ have negative real part. So it suffices to show that $F$ is analytically continuable in $z=0$.

1. case: $B$ has no transient indices. So the set $R$ of recurrent indices comprises the whole index set. By suitable permutation of $R$ we have block diagonal form $B=B_{R}=\textnormal{diag}(B_{1},...,B_{k})$ with irreducible intensity matrices $B_{i}$ as main diagonal submatrices of $B$ and with all other submatrices besides the $B_{i}$ blockwise equal to zero. Due to Lemma 4 the functions $z\mapsto|zI_{i}-B_{i}|$, with identity matrix $I_{i}$ of appropriate dimension, have the simple root $z=0$. So each entry of the matrix

[TABLE]

has pole order at most one at $z=0$. So by multiplication with $z$ we obtain $F(z)$ with pole order at most zero at $z=0$.

2. case: The set $T$ of transient indices is not empty. Then we have the additional submatrix $B_{T}$ as the lower right block of $B$ and the lower left block $B_{T,R}$. Since $B_{T}$ is invertible due to Lemma 4 the entries of $C(z):=zI_{T}-B_{T}$ have no pole at $z=0$. With $A(z):=zI_{R}-B_{R}$ it holds

[TABLE]

So after multiplication with $z$ we have pole order at most zero at $z=0$. ∎

3 Computation of the final limit

We show how to compute the final limit of Theorem 1 via matrix operations.

Theorem 6**.**

For a right intensity matrix $B$ with finite index set $Z$ the final limit $P$ is stochastic. For every recurrence class $J\subset Z$ with respect to $B$ a row of $P$ with index in $J$ equals the unique row vector $p_{J}:=(p_{j})_{j\in Z}$ defined by $p_{J}B=0$, $p_{j}>0$ for $j\in J$, $p_{j}=0$ for $j\in Z\setminus J$ and $\sum\limits_{j\in Z}p_{j}=1$. So the stochastic submatrix $P_{J}=\lim\limits_{t\to\infty}\exp(tB_{J})$ has equal rows with positive entries. In case the set $T\subset Z$ of transient indices with respect to $B$ is not empty the set $R:=Z\setminus T$ of recurrent indices is not empty, too. The matrices $P_{T}$ and $B_{T,R}P_{R}+B_{T}P_{T,R}$ are zero. Hereby $B_{T}$ is invertible, whence

[TABLE]

For the column vector $(f_{i,J})_{i\in T}:=-B_{T}^{-1}\sum\limits_{j\in J}(b_{ij})_{i\in T}$ and the set $\mathfrak{C}$ of recurrence classes the row of index $i\in T$ of $P=(p_{ij})$ equals $\sum\limits_{J\in\mathfrak{C}}f_{i,J}p_{J}$. Hence we have $p_{ij}=f_{i,J}p_{jj}$ for $i\in T$ and $j\in J\in\mathfrak{C}$.

Proof.

Due to Lemma 3 the transition matrix $Y(t):=\exp(tB)$ is stochastic. By the limiting process with $t\to\infty$ which is allowed by Theorem 1 this implies the first assertion. From the definition of a recurrence class $J$ it follows $b_{ij}=0$ for $i\in J$ and $j\in Z\setminus J$. So it holds $B^{n}_{J}=(B_{J})^{n}$ and $B^{n}_{J,Z\setminus J}=O$ for all $n\in\mathbb{N}_{0}$. This implies $Y(t)_{J}=\exp(tB_{J})\textnormal{ and }Y(t)_{J,Z\setminus J}=O$ for all $t\in\mathbb{R}$, hence $P_{J}=\lim\limits_{t\to\infty}\exp(tB_{J})$ and $P_{J,Z\setminus J}=O$. Therefore $P_{J}$ is also stochastic. Since $B_{J}$ is irreducible the left null space of $B_{J}$ is spanned by a vector $p_{J}=(p_{j})_{j\in J}$ with positive coordinates $p_{j}$ according to Lemma 4. Without loss of generality we may assume $\sum\limits_{j\in J}p_{j}=1$. By Lemma 2 it holds $P_{J}B_{J}=O$. So $p_{J}$ coincides with each row of $P_{J}$. By filling with zeros at indices $j\in Z\setminus J$ the row vector $p_{J}$ becomes $p_{J}$ like in the second assertion. For $T\neq\emptyset$ we obtain the block form $B=\left(\begin{matrix}B_{R}&\ O\\ B_{T,R}&\ B_{T}\end{matrix}\right)$ by permutation of the index set. By Lemma 4 it holds $P_{T}B_{T}=O$ and $B_{T}$ is invertible. So we have $P_{T}=O$. Induction on $n\in\mathbb{N}$ yields ${B^{n+1}}_{T,R}=B_{T,R}B_{R}^{n}+B_{T}{B^{n}}_{T,R}$. So for every $t\in\mathbb{R}$ it holds

[TABLE]

by Lemma 2. Because of $I_{T,R}=O$ it follows

[TABLE]

By the limiting process with $t\to\infty$ this implies the formula for $P_{T,R}$ since $\lim\limits_{t\to\infty}Y^{\prime}(t)=O$ by Lemma 2. Let $j\in J\in\mathfrak{C}$. As shown above it holds $p_{ij}=p_{jj}$ for $i\in J$ and $p_{ij}=0$ for $i\in R\setminus J$. This implies

[TABLE]

By cancelling out $B_{T}$ we obtain $p_{ij}=f_{i,J}p_{jj}$ for $i\in T$ and hence the identity for the $i$-th row of $P$.333For the embedded Markov chain mentioned in the latter footnote $f_{i,J}$ is the enter probability from the transient state $i$ into the recurrence class $J$. ∎

Remark 7*.*

The theorem suggests the following pseudo-algorithm for computing the final limit $P$ from right intensity matrix $B$ with index set $Z$:

Initialise $P:=O$ (zero matrix of dimension like $B$) and $T:=Z$.

• 1.

Compute the set $\mathfrak{C}$ of recurrence classes with respect to $B$.

• 2.

For $J\in\mathfrak{C}$:

• 2.1.

Find a non-zero element $x=(x_{j})_{j\in J}$ of the left null space of $B$.

• 2.2.

Define $p:=\lambda^{-1}x$ with $\lambda:=\sum\limits_{j\in J}x_{j}$.

• 2.3.

Substitute every row of $P_{J}$ by $p$.

• 2.2.

Substitute $T$ by $T\setminus J$.

• 3.

If $T$ is not empty:

• 3.1.

For $J\in\mathfrak{C}$:

• 3.1.1.

Compute $(f_{i,J})_{i\in T}:=-B_{T}^{-1}B_{T,J}(1...1)^{t}$.

• 3.1.2.

For $i\in T$: For $j\in J$: Substitute $p_{ij}$ by $f_{i,J}p_{jj}$.

• 4.

Output $P$.

For a square matrix $A$ with index set $Z$ of $n>1$ elements let

[TABLE]

Then index $i$ is in the same recurrence class as index $j$ if and only if $c_{ij}$ and $c_{ji}$ do not vanish. This explains how to do step 1.