Absorption time and absorption probabilities for a family of multidimensional gambler models

TL;DR

This paper derives formulas for winning probabilities and game duration distributions in a family of multidimensional gambler models, extending known results from one-dimensional cases using advanced probabilistic and matrix techniques.

Contribution

It introduces new formulas and sample-path constructions for multidimensional gambler models, leveraging intertwining matrices, Siegmund duality, and Kronecker products.

Findings

Explicit formulas for winning probabilities in multidimensional models

Distribution of game duration expressed via eigenvalues of related one-dimensional games

Sample-path constructions for game duration in many cases

Abstract

For a family of multidimensional gambler models we provide formulas for the winning probabilities (in terms of parameters of the system) and for the distribution of game duration (in terms of eigenvalues of underlying one-dimensional games). These formulas were known for one-dimensional case - initially proofs were purely analytical, later probabilistic construction has been given. Concerning the game duration, in many cases our approach yields sample-path constructions. We heavily exploit intertwining between (not necessary) stochastic matrices (for game duration results), a notion of Siegmund duality (for winning/ruin probabilities), and a notion of Kronecker products.

Equations

$$\left\{\begin{array}[]{llllllll}p_{j}(i_{j})&\textrm{if}&i_{j}^{\prime}=i_{j}+1,i_{k}^{\prime}=i_{k},k\neq j,\\[3.0pt] q_{j}(i_{j})&\textrm{if}&i_{j}^{\prime}=i_{j}-1,i_{k}^{\prime}=i_{k},k\neq j,\\[3.0pt] \sum_{j:i_{j}=1}q_{j}(1)&\textrm{if}&(i^{\prime}_{1},\ldots,i^{\prime}_{j})=-\infty,\\[3.0pt] 1-\sum_{k=1}^{d}(p_{k}(i_{k})+q_{k}(i_{k}))&\textrm{if}&i_{j}^{\prime}=i_{j},1\leq j\leq d,\\[3.0pt] 1&\textrm{if}&(i_{1},\ldots,i_{j})=(i^{\prime}_{1},\ldots,i^{\prime}_{j})=-\infty.\end{array}\right.$$

$$\rho ((i_1,\dots ,i_d))=P({\tau }_{(N_1,\dots ,N_d)}<{\tau }_{-\infty }|Z_0=(i_1,\dots ,i_d)),$$

$$\rho ((i_1,\dots ,i_d))=\frac{\prod _{j=1}^d\left(\sum _{n_j=1}^{i_j}\prod _{r=1}^{n_j}\left(\frac{q_j(r)}{p_j(r)}\right)\right)}{\prod _{j=1}^d\left(\sum _{n_j=1}^{N_j}\prod _{r=1}^{n_j}\left(\frac{q_j(r)}{p_j(r)}\right)\right)}.$$

$$\displaystyle\mathbf{P}_{Y}(i,i^{\prime})=\left\{\begin{array}[]{llllllll}p(i)&\textrm{if}&i^{\prime}=i+1,\\[3.0pt] q(i)&\textrm{if}&i^{\prime}=i-1,\\[3.0pt] 1-(p(i)+q(i))&\textrm{if}&i^{\prime}=i.\\[3.0pt] \end{array}\right.$$

$$pg{f}_{T_{1,N}}(u):=\mathbf{E}{u}^{T_{1,N}}=\prod _{k=1}^{N-1}\left[\frac{(1-{\lambda }_k)u}{1-{\lambda }_{k}u}\right],$$

$$\displaystyle\mathbf{P}_{\hat{X}}(i,i^{\prime})=\left\{\begin{array}[]{llllllll}1-\lambda_{i}&\mathrm{if}\ i^{\prime}=i+1,\\[3.0pt] \lambda_{i}&\mathrm{if}\ i^{\prime}=i,\\[3.0pt] 0&\mathrm{otherwise}.\\[3.0pt] \end{array}\right.$$

$$pg{f}_{T_{s,N}}(u):=\mathbf{E}{u}^{T_{s,N}}=\frac{\prod _{k=1}^{N-1}\left[\frac{(1-{\lambda }_k)u}{1-{\lambda }_{k}u}\right]}{\prod _{k=1}^{s-1}\left[\frac{(1-{\lambda }_k^{\lfloor s\rfloor })u}{1-{\lambda }_k^{\lfloor s\rfloor }u}\right]},$$

$$\displaystyle\mathbf{P}_{Y}^{\lfloor s\rfloor}(i,i^{\prime})=\left\{\begin{array}[]{llllllll}p(i)&\textrm{if}&i^{\prime}=i+1,1\leq i\leq s-2,\\[3.0pt] q(i)&\textrm{if}&i^{\prime}=i-1,2\leq i\leq s-1\\[3.0pt] 1-(p(i)+q(i))&\textrm{if}&i^{\prime}=i,1\leq i\leq s-1.\\[3.0pt] \end{array}\right.$$

$$pg{f}_{T_{s,N}}(u)=\mathbf{E}{u}^{T_{s,N}}=$$

$$pg{f}_{T_{s,0}}(u)=\mathbf{E}{u}^{T_{s,0}}=$$

$$\displaystyle\mathbf{P}_{Y}^{\lceil s\rceil}(i,i^{\prime})=\left\{\begin{array}[]{llllllll}p(i)&\textrm{if}&i^{\prime}=i+1,s+1\leq i\leq N-2,\\[3.0pt] q(i)&\textrm{if}&i^{\prime}=i-1,s+2\leq i\leq N-1\\[3.0pt] 1-(p(i)+q(i))&\textrm{if}&i^{\prime}=i,s+1\leq i\leq N-1.\\[3.0pt] \end{array}\right.$$

$$\mathbf{A}\otimes \mathbf{B}=\left[\begin{array}{cccc}{a}_{11}\mathbf{B}& a_{12}\mathbf{B}& \dots & a_{1m}\mathbf{B}\\ a_{21}\mathbf{B}& a_{22}\mathbf{B}& \dots & a_{2m}\mathbf{B}\\ 4& & & \\ a_{n1}\mathbf{B}& a_{n2}\mathbf{B}& \dots & a_{nm}\mathbf{B}\end{array}\right]$$

$$\mathbf{A}\oplus \mathbf{B}=\mathbf{A}\otimes {\mathbf{I}}_{\mathbf{B}}+{\mathbf{I}}_{\mathbf{A}}\otimes \mathbf{B},$$

$$\underset{i=1}{\overset{n}{⨂}}{\mathbf{A}}_i=(\dots (({\mathbf{A}}_1\otimes {\mathbf{A}}_2)\otimes {\mathbf{A}}_3)\dots )\otimes {\mathbf{A}}_n={\mathbf{A}}_1\otimes {\mathbf{A}}_2\otimes \dots \otimes {\mathbf{A}}_n$$

$$\underset{i=1}{\overset{n}{⨁}}{\mathbf{A}}_i=(\dots (({\mathbf{A}}_1\oplus {\mathbf{A}}_2)\oplus {\mathbf{A}}_3)\dots )\oplus {\mathbf{A}}_n={\mathbf{A}}_1\oplus {\mathbf{A}}_2\oplus \dots \oplus {\mathbf{A}}_n.$$

$$\mathbf{P}_{Y}(i,j)=\left\{\begin{array}[]{llllllll}\mathbf{P}^{\prime}_{Y}(\mathbf{e}_{i},\mathbf{e}_{j})&\textrm{if}&\mathbf{e}_{i},\mathbf{e}_{j}\in\mathbb{E},\\[3.0pt] 1-\sum_{\mathbf{e}_{k}\in\mathbb{E}^{\prime}}\mathbf{P}^{\prime}_{Y}(\mathbf{e}_{i},\mathbf{e}_{k})&\textrm{if}&\mathbf{e}_{i}\in\mathbb{E}^{\prime},\mathbf{e}_{j}=\mathbf{e}_{0},\\[3.0pt] 1&\textrm{if}&\mathbf{e}_{i}=\mathbf{e}_{j}=\mathbf{e}_{0}.\\[3.0pt] 0&\textrm{if}&\mathbf{e}_{i}=\mathbf{e}_{0},\mathbf{e}_{j}\in\mathbb{E}.\\[3.0pt] \end{array}\right.$$

$${\rho }_j^{(k)}(i)=P({\tau }_{N_j}<{\tau }_0|Z_j^{(k)}(0)=i)={\rho }_j(i).$$

$$\mathbf{R}^{\prime}_{Z_{j}^{(k)}}=\left\{\begin{array}[]{lll}\mathbf{P}^{\prime}_{Z_{j}^{(k)}}&\mathrm{if\ }j\in A_{k},\\[10.0pt] \mathbf{I}_{j}&\mathrm{if\ }j\notin A_{k},\\ \end{array}\right.$$

$${\mathbf{P}}_Z^{\prime }=\sum _{k=1}^m{\mathbf{B}}_k\left(\underset{j\le d}{⨂}{\mathbf{R}}_{Z_j^{(k)}}^{\prime }\right)$$

$$\rho (i_1,\dots ,i_d)=\prod _{j=1}^d{\rho }_j(i_j).$$

$${\rho }_j(i_j)=\frac{\sum _{n_j=1}^{i_j}\prod _{r=1}^{n_j-1}\left(\frac{q_j(r)}{p_j(r)}\right)}{\sum _{n_j=1}^{N_j}\prod _{r=1}^{n_j-1}\left(\frac{q_j(r)}{p_j(r)}\right)}.$$

$$\mathbf{R}_{X_{j}^{*(k)}}^{\prime}=\left\{\begin{array}[]{lll}\mathbf{P}_{X^{*}_{j}}^{\prime}&\mathrm{if\ }j\in A_{k},\\[10.0pt] \mathbf{I}_{j}&\mathrm{if\ }j\notin A_{k},\\ \end{array}\right.$$

$${\mathbf{P}}_{X^{\ast }}^{\prime }=\sum _{k=1}^m{b}_k\left(\underset{j\le d}{⨂}{\mathbf{R}}_{X_j^{\ast (k)}}^{\prime }\right)$$

$$pg{f}_{T_{{\nu }^{\ast },\mathbf{N}}^{\ast }}(s)=\sum _{\hat{\mathbf{e}}\in \mathbb{E}}\hat{\nu }(\hat{\mathbf{e}})pg{f}_{{\hat{T}}_{\hat{\mathbf{e}},\mathbf{N}}}(s)\left(\prod _{j=1}^d{\rho }_j(1)\right),$$

$$\hat{\nu }={\nu }^{\ast }\underset{j\le d}{⨂}{\Lambda }_j^{-1},$$

$$\left\{\begin{array}[]{lll}\displaystyle\prod_{j\in B}\left(1-\lambda_{i_{j}}^{(j)}\right)\sum_{k:B\subseteq A_{k}}\left(b_{k}\prod_{j\in A_{k}\setminus B}\lambda_{i_{j}}^{(j)}\right)&\textrm{if }\ i^{\prime}_{j}=i_{j}+1,\\ &\quad j\in B\subseteq\{1,\ldots,d\},B\neq\emptyset\\[16.0pt] \displaystyle\sum_{k=1}^{m}b_{k}\prod_{j\in A_{k}}\lambda_{i_{j}}^{(j)}&\textrm{if }\ i^{\prime}_{j}=i_{j}\textrm{ for }j=1,\ldots,d.\\[18.0pt] 0&\textrm{otherwise}.\end{array}\right.$$

$$\forall (\mathbf{e}\in \mathbb{E}){\nu }^{\ast }(\mathbf{e})\ne 0\Rightarrow \exists ({\mathbf{e}}^{\prime }⪰\mathbf{e})\hat{\nu }({\mathbf{e}}^{\prime })>0$$

$$pg{f}_{T_{{\nu }^{\ast },\mathbf{N}}^{\ast }}(s)=\sum _{\hat{\mathbf{e}}\in \mathbb{E}}\hat{\nu }(\hat{\mathbf{e}})pg{f}_{{\hat{T}}_{\hat{\mathbf{e}},\mathbf{N}}}(s).$$

$$T^{*}_{(1,\ldots,1),\mathbf{N}}=\left\{\begin{array}[]{llll}\hat{T}_{(1,\ldots,1),\mathbf{N}}&\mathrm{with\ probability\ }\prod_{j=1}^{d}\rho_{j}(1),\\[8.0pt] +\infty&\mathrm{with\ probability\ }1-\prod_{j=1}^{d}\rho_{j}(1).\\ \end{array}\right.$$

$${\hat{X}}_0={\hat{\mathbf{e}}}_0\text{ with probability }\frac{\hat{\nu }({\hat{\mathbf{e}}}_0)\Lambda ({\hat{\mathbf{e}}}_0,{\mathbf{e}}_0^{\ast })}{{\nu }^{\ast }({\mathbf{e}}_0^{\ast })}.$$

$${\hat{X}}_n={\hat{\mathbf{e}}}_n\text{ with probability }\frac{{\mathbf{P}}_{\hat{X}}({\hat{\mathbf{e}}}_{n-1},{\hat{\mathbf{e}}}_n)\Lambda ({\hat{\mathbf{e}}}_n,{\mathbf{e}}_n^{\ast })}{({\mathbf{P}}_{X^{\ast }}\Lambda )({\hat{\mathbf{e}}}_{n-1},{\mathbf{e}}_n^{\ast })}.$$

Full text

\SHORTTITLE

Multidimensional gambler models \TITLEAbsorption time and absorption probabilities for a family of multidimensional gambler models††thanks: Work supported by NCN Research Grant DEC-2013/10/E/ST1/00359. \AUTHORSPaweł Lorek111Mathematical Institute, Wrocław University, Wrocław, Poland. \EMAILPawel.Lorek@math.uni.wroc.pl, Piotr.Markowski@math.uni.wroc.pl and Piotr Markowski22footnotemark: 2\KEYWORDSGeneralized gambler’s ruin problem ; absorption probability ; absorption time ; intertwining; eigenvalues; Siegmund duality; partial ordering; Kronecker products; Möbius monotonicity \AMSSUBJ60J10 \AMSSUBJSECONDARY 60G40; 60J80 \SUBMITTEDJuly 18, 2018 \ACCEPTED… \VOLUME0 \YEAR2018 \PAPERNUM0 \DOI……… \ABSTRACTFor a family of multidimensional gambler models we provide formulas for the winning probabilities (in terms of parameters of the system) and for the distribution of game duration (in terms of eigenvalues of underlying one-dimensional games). These formulas were known for one-dimensional case - initially proofs were purely analytical, later probabilistic construction has been given. Concerning the game duration, in many cases our approach yields sample-path constructions. We heavily exploit intertwining between (not necessary) stochastic matrices (for game duration results), a notion of Siegmund duality (for winning/ruin probabilities), and a notion of Kronecker products.

1 Introduction

In the one-dimensional gambler’s ruin problem two players start a game with a total amount of, say, $N$ dollars and initial values $k$ and $N-k$. At each step they flip the coin (not necessary unbiased) to decide who wins a dollar. The game is over when one of them goes bankrupt. There are some fundamental questions related to this process.

• Q1

Starting with $i$ dollars, what is the probability of winning?

• Q2

Starting with $i$ dollars, what is the distribution (or structure) of game duration (i.e., absorption time)? Or, what is the distribution (or structure) of game duration conditioned on winning/losing?

In this paper we will answer above questions for a wide class of multidimensional generalizations of gambler’s ruin problem. The proofs will be probabilistic in most cases, utilizing either Siegmund duality or intertwining between chains.

Generalized multidimensional gambler models

In [11] we considered the following generalization. There is one player (referred as “we”) playing with $d\geq 1$ other players. Our initial assets are $(i_{1},\ldots,i_{d})$ and assets of consecutive players are $(N_{1}-i_{1},\ldots,N_{d}-i_{d})$ ($N_{j}\geq 1$ is a total amount of assets with player $j$). Then, with probability $p_{j}(i_{j})$ we win one dollar with player $j$ and with probability $q_{j}(i_{j})$ we lose it. With the remaining probability $1-\sum_{k=1}^{d}(p_{k}(i_{j})+q_{j}(i_{k}))$ we do nothing (i.e., ties are also possible). Once we win completely with player $j$ (i.e., $i_{j}=N_{j}$) we do not play with him/her anymore. We lose the whole game if we lose with at least one player, i.e., when $i_{j}=0$ for some $j=1,\ldots,d$. The game can be described more formally as a Markov chain $Z$ with two absorbing states. The state space is $\mathbb{E}=\{(i_{1},\ldots,i_{d}):1\leq i_{j}\leq N_{j},1\leq j\leq d\}\cup\{-\infty\}$ (where $-\infty$ means we lose). For convenience denote $p_{j}(N_{j})=q_{j}(N_{j})=0,j=1,\ldots,d$. Assume that for all $i_{j}\in\{1,\ldots,N_{j}\},j\in\{1,\ldots,d\}$ we have $p_{j}(i_{j})>0,q_{j}(i_{j})>0$ and $\sum_{k=1}^{d}(p_{k}(i_{k})+q_{k}(i_{k}))\leq 1.$ With some abuse of notation, we will sometimes write $(i^{\prime}_{1},\ldots,i^{\prime}_{d})=-\infty$. The transitions of the described chain are the following:

$\displaystyle\mathbf{P}_{Z}((i_{1},\ldots,i_{j}),(i^{\prime}_{1},\ldots,i^{\prime}_{j}))=$

[TABLE]

The chain has two absorbing states: $(N_{1},\ldots,N_{d})$ (we win) and $-\infty$ (we lose). Let

[TABLE]

where $\tau_{(i_{1}^{\prime},\ldots,i_{d}^{\prime})}:=\inf\{n\geq 0:Z_{n}={(i_{1}^{\prime},\ldots,i_{d}^{\prime})}\}$. Roughly speaking, $\rho((i_{1},\ldots,i_{d}))$ is the probability of winning starting at $(i_{1},\ldots,i_{d})$. In [11] we derived the formula for this probability, namely

[TABLE]

In this paper we consider a much wider class of $d$-dimensional games - the chain given in (1) is just a special case. For example, within the class we can win/lose in one step with many players. The multidimensional chain is constructed from a variety of one-dimensional chains using Kronecker products. For this class:

• •

We give expressions for the winning probabilities and prove that it is a product of the winning probabilities corresponding to one-dimensional games. In particular, for a subclass of the multidimensional chains, constructed from one-dimensional birth and death chains, the winning probabilities are given in (3). The main tool for showing winning probabilities is the Siegmund duality defined for partially ordered state spaces, exploiting the results from [12].

• •

We give formulas for the distributions of game duration. In some cases a probability generating function is given, in other cases we show that the absorption time is equal, in distribution, to the absorption time of another chain, which is, in a sense, a multidimensional pure-birth chain. In many cases, the probabilistic proof is given. To show the absorption distribution, we exploit the spectral polynomials given in [5], and their variations considered in [7], [13].

Remark 1.1**.**

In [11] we considered the chain – given in (1) – which is constructed from $d$ one-dimensional birth and death chains in a very specific way. The method from this article is much more general, we can construct many different multidimensional chains from given $d$ one-dimensional birth and death chains. It is worth mentioning, that even for the case (1), the proof is quite different (from the one in [11]).

Several variations (including multidimensional ones) of gambler’s ruin problem have been considered. Researchers usually study absorption probabilities, absorption time, or both. In [9] authors consider two-dimensional model (they consider two currencies) and study expected game duration. In [16] some multidimensional game is considered: at each step two players are randomly chosen, these players play a regular game, all till one of the players have all the coins. Author derives the probability that specific player wins, the expected number of turns in total and between two given players. In [15] the following multidimensional game is considered: there are $n$ players, at each step there is one winner which collects $n-1$ coins from other players, whereas all others lose $1$ coin. Asymptotic probability for individual ruin and dependence of ruin time are studied. In [18] the multidimensional case is considered, in which with equal probability a unit displacement in any direction is possible. Moments of leaving some a ball are considered. In [3] authors present a new probabilistic analysis of distributed algorithm re-considering a variation of banker algorithm. Mathematically, it is random walk on a rectangle with specified absorbing states. The results are generalized to the case with many players and resources.

The absorption probability of given chain may be related to the stationary distribution of some ergodic chain. This relation is given using Siegmund duality, the notion introduced in [17]. This is also the tool we use for showing absorption probabilities. Already in [10] similar duality between some random walks on integers was shown. It was also studied in financial mathematics, where the probability that a dual risk process starting at some level is ruined, is equal to the probability that the stationary queue length exceeds this level (see [1], [2]). In all these cases the Siegmund duality was defined for linear ordering of the state space. The existence of a Siegmund dual for linearly ordered state space requires stochastic monotonicity of a chain. Recently, in [12] we provided if and only if conditions for existence of Siegmund dual for partially ordered state spaces (roughly speaking, the Möbius monotonicity is required). In this paper, we exploit this duality defined for a coordinate-wise partial ordering.

Absorption time

Consider one-dimensional game corresponding to the gambler’s ruin problem. Let $N$ be a total amount of money. Being at state $i\in\{2,\ldots,N-1\}$ we can either win one dollar with probability $p(i)>0$ or lose it with probability $q(i)>0$, with the remaining probability nothing happens. Assuming $p(1)>0$ and $p(N)=q(N)=p(0)=q(0)=0$ the transitions are the following:

[TABLE]

States 0 and $N$ are absorbing. Consider two cases:

Case: $q(1)=0$

Roughly speaking, if started in $i\geq 1$ the chain never reaches [math] and this is actually a birth and death chain on $\{1,\ldots,N\}$ with $N$ being the only absorbing state. Define $T_{a,b}=\inf\{n\geq 0:Y_{n}=b\ |Y_{0}=a\ \}$. A well known theorem attributed to Keilson [8] states, that the probability generating function $pgf$ of $T_{1,N}$ is the following:

[TABLE]

where $-1\leq\lambda_{k}<1,k=1,\ldots,N-1$ are $N-1$ non-unit eigenvalues of $\mathbf{P}_{Y}$. The proof was purely analytical. Note that (5) corresponds to the sum of $N_{j}$ geometric random variables, provided that all eigenvalues are positive (which, in this case, is equivalent to the stochastic monotonicity of the chain). For this case, Fill [5] (in 2009) gave a probabilistic proof of (5) using strong stationary duality and intertwinings between chains. Note that in this case (5) can be rephrased as:

Theorem 1.2**.**

Let $X^{*}$ be an absorbing chain on $\mathbb{E}=\{1,\ldots,N\}$ starting at $1$ with transition matrix $\mathbf{P}_{X^{*}}$ given in (4) having positive eigenvalues $\lambda_{k}>0,k=1,\ldots,N$. Then $T^{*}_{1,N}$ has the same distribution as $\hat{T}_{1,N}$, the absorption time of $\hat{X}$ on $\hat{\mathbb{E}}=\mathbb{E}$ starting at $1$ with transition matrix

[TABLE]

The chain $Y$ on $\{1,\ldots,N\}$ is called pure birth if $\mathbf{P}_{Y}(i,j)=0$ for $j\leq i$. Similarly, a multidimensional chain $Y$ on $\mathbb{E}=\{(i_{1},\ldots,i_{d}):1\leq i_{j}\leq N_{j},1\leq j\leq d\}$ ($N_{j}$ can be $\infty$) is said to be pure birth if the probability of decreasing any set of coordinates at one step is 0.

Simply noting that for any $1<s<N$ we have $T_{1,N}=T_{1,s}+T_{s,N}$ and that $T_{1,s}$ and $T_{s,N}$ are independent (derived in 2012, see Cor. 2.1 [7] for continuous time version) we have

[TABLE]

where $\lambda_{k}^{\lfloor i\rfloor}$ are the eigenvalues of substochastic $(s-1)\times(s-1)$ matrix

[TABLE]

Case: $q(1)>0$

In this case, authors in [7] (different proof is given in [13]) derived formulas for $pgf$ of ${T_{s,N}}$ and ${T_{s,0}}$ (more precisely, they derived formulas for continuous time versions), which, in discrete case, are given by

[TABLE]

where $\rho(s)$ is the probability of winning (i.e., (2) with $d=1,i_{1}=s$) and $\lambda^{\lceil s\rceil}_{k}$ are the eigenvalues of substochastic matrix (of the size $N-s-1$)

[TABLE]

In this paper we aim at presenting results similar to Theorem 1.2 and to (7) for a wide class of multidimensional extensions of gambler’s ruin problem.

2 Kronecker product and main results

To state our main results we need to recall a notion of Kronecker product. Let $\mathbf{A}$ be a matrix of size $n\times m$. Then, for any matrix $\mathbf{B}$ the Kronecker product of the matrices is defined as follows:

[TABLE]

For square matrices $\mathbf{A}$ and $\mathbf{B}$ it is also convenient to define the Kronecker sum as:

[TABLE]

where $\mathbf{I}_{\mathbf{A}}$ ($\mathbf{I}_{\mathbf{B}}$) is the identity matrix of the same size as $\mathbf{A}$ ($\mathbf{B}$).

Both, product and sum, are extended as:

[TABLE]

and

[TABLE]

Notation

For a convenience, for given substochastic matrix $\mathbf{P}^{\prime}_{Y}$ on $\mathbb{E}^{\prime}=\{\mathbf{e}_{1},\ldots,\mathbf{e}_{M}\}$ by $\mathbf{P}_{Y}=\mathcal{F}_{\mathbf{e}_{0}}(\mathbf{P}^{\prime}_{Y})$ we denote a stochastic matrix on $\mathbb{E}=\{\mathbf{e}_{0}\}\cup\mathbb{E}^{\prime}$ constructed from $\mathbf{P}^{\prime}_{Y}$ in the following way:

[TABLE]

Similarly, for a stochastic matrix $\mathbf{P}_{Y}$ on $\mathbb{E}=\{\mathbf{e}_{0}\}\cup\mathbb{E}^{\prime}$ let $\mathbf{P}^{\prime}_{Y}=\mathcal{F}^{-1}_{\mathbf{e}_{0}}(\mathbf{P}_{Y})$ be a substochastic matrix on $\mathbb{E}^{\prime}$ resulting from $\mathbf{P}_{Y}$ by removing row and column corresponding to the state $\mathbf{e}_{0}$.

For a Markov chain $Y$ on $\mathbb{E}=\{\mathbf{e}_{1},\ldots,\mathbf{e}_{M}\}$ we say that $A\subseteq\mathbb{E}$ is a communication class if for all $\mathbf{e},\mathbf{e}^{\prime}\in A$ we have $\mathbf{P}_{Y}^{n}(\mathbf{e},\mathbf{e}^{\prime})>0$ for some $n\geq 0$.

For given chain $Y$ define $T_{\nu,\mathbf{e}^{\prime}}:=\inf\{n\geq 0:Y_{n}=\mathbf{e}^{\prime}|Y_{0}\sim\nu\}$. Slightly abusing the notation, by $T_{\mathbf{e},\mathbf{e}^{\prime}}$ we mean $T_{\nu,\mathbf{e}^{\prime}}$ with $\nu=\delta_{\mathbf{e}}$.

For $\mathbb{E}=\{\mathbf{e}_{1},\ldots,\mathbf{e}_{M}\}$ and for $f:\mathbb{E}\to\mathbb{R}$, we define a row vector $\boldsymbol{f}=(f(\mathbf{e}_{1}),\ldots,f(\mathbf{e}_{M}))$.

For $N_{j}>0,j=1,\ldots,d$ define $\mathbf{N}=(N_{1},\ldots,N_{d})$.

Now we are ready to state our main results.

2.1 Absorption probabilities

Theorem 2.1**.**

Fix integers $d\geq 1,m\geq 1$. For $k=1,\ldots,m$ let $A_{k}\subseteq\{1,\ldots,d\}$. Assume

• •

$\forall(1\leq k\leq m)$* $\mathbf{P}_{Z^{(k)}_{j}}=\mathcal{F}_{0}(\mathbf{P}^{\prime}_{Z^{(k)}_{j}})$ is a stochastic matrix corresponding to a Markov chain $Z_{j}^{(k)}$ on $\mathbb{E}_{j}=\{0,1,\ldots,N_{j}\}$ such that for $i\in\mathbb{E}_{j}$ we have*

[TABLE]

In other words, $Z_{j}^{(k)}$ are $m$ ($k=1,\ldots,m$) chains having the same winning probability at every state $i$.

• •

Let

[TABLE]

where $\mathbf{I}_{j}$ is the identity matrix of size $N_{j}\times N_{j}$.

• •

Let $\mathbf{B}_{i},i=1,\ldots,m$ be either

• –

any real numbers (i.e., $\mathbf{B}_{k}\in\mathbb{R}$) such that $\sum_{k=1}^{m}\mathbf{B}_{k}=1$, or

• –

square matrices of size $\prod_{j=1}^{d}N_{j}\times\prod_{j=1}^{d}N_{j}$ such that $\sum_{k=1}^{m}\mathbf{B}_{k}=\mathbf{I}$ (identity matrix of the appropriate size)

• •

The matrix $\mathbf{P}_{Z}=\mathcal{F}_{-\infty}(\mathbf{P}_{Z^{\prime}})$ with

[TABLE]

is stochastic on $\mathbb{E}=\{-\infty\}\cup\bigotimes_{j\leq d}\mathbb{E}_{j}^{\prime},$ set $\mathbb{E}\setminus\{\{\mathbf{N}\}\cup\{-\infty\}\}$ is a communication class.

Then, the winning probability (i.e., absorption at $\mathbf{N}$) of the Markov chain $\mathbf{Z}$ on $\mathbb{E}=\{-\infty\}\cup\{1,\ldots,N_{1}\}\times\ldots\times\{1,\ldots,N_{d}\}$ with transition matrix $\mathbf{P}_{Z}=\mathcal{F}_{-\infty}(\mathbf{P}^{\prime}_{Z})$ is given by

[TABLE]

The proof is postponed to Section 4.2.

Note that $\mathbf{P}_{Z_{j}^{(k)}}$ in Theorem 2.1 are general. If we only know the winning probabilities of $\mathbf{P}_{Z_{j}^{(k)}}$ (they cannot depend on $k$), then we know the winning probabilities of $Z$. Taking $\mathbf{P}_{Z_{j}^{(k)}}$ corresponding to gambler’s ruin game given in (4) we have:

Corollary 2.2**.**

Let $\mathbf{P}_{Z_{j}^{(k)}}$ for $j=1,\ldots,d$ be the birth and death chain given in (4). Then, the winning probability of $\mathbf{P}_{Z}=\mathcal{F}_{-\infty}(\mathbf{P}^{\prime}_{Z})$ is given by (3).

Proof 2.3**.**

For $\mathbf{P}_{Z_{j}^{(k)}}$ the winning probability is known (shown in (18)), it is

[TABLE]

Assertion of Theorem 2.1 completes the proof.

The chain $\mathbf{Z}$ can be interpreted as $d$-dimensional game, with state $(N_{1},\ldots,N_{d})$ corresponding to winning and state $-\infty$ corresponding to losing.

2.2 Absorption time

We have the following extension of Theorem 1.2 to the multidimensional case:

Theorem 2.4**.**

Fix integers $d\geq 1,m\geq 1$. For $k=1,\ldots,m$ let $A_{k}\subseteq\{1,\ldots,d\}$. Let $b_{i}\in\mathbb{R},i=1,\ldots,m$ such that $\sum_{k=1}^{m}b_{i}=1$. Let, for $1\leq j\leq d$, $\mathbf{P}_{X^{*}_{j}}$ be a stochastic matrix corresponding to a birth and death chain $X_{j}^{*}$ on $\mathbb{E}_{j}=\{0,\ldots,N_{j}\}$ with transitions given in (4) with birth rates $p_{j}(i)$ and death rates $q_{j}(i)$. Let, for $1\leq j\leq d$, $\mathbf{P}_{X^{*}_{j}}^{\prime}=\mathcal{F}^{-1}_{0}(\mathbf{P}_{X^{*}_{j}})$ be a substochastic matrix on $\mathbb{E}_{j}^{\prime}=\{1,\ldots,N_{j}\}$ and

[TABLE]

where $\mathbf{I}_{j}$ is the identity matrix of size $N_{j}\times N_{j}$. I.e., $\mathbf{R}_{X^{*(k)}_{j}}^{\prime}$ is either matrix $\mathbf{P}_{X^{*}_{j}}^{\prime}$ or an identity matrix. Let $\lambda_{1}^{(j)}\leq\ldots\leq\lambda_{N_{j}-1}^{(j)}<\lambda_{N_{j}}^{(j)}=1$ be the eigenvalues of $\mathbf{P}^{\prime}_{X^{*}_{j}}$.

Assume

• A1

The chains $\mathbf{P}_{X^{*}_{j}},j=1,\ldots,d$ are stochastically monotone.

• A2

The matrix $\mathbf{P}_{X^{*}}=\mathcal{F}_{-\infty}(\mathbf{P}_{X^{*}}^{\prime})$ with

[TABLE]

is stochastic matrix on $\mathbb{E}=\{-\infty\}\cup\bigotimes_{j\leq d}\mathbb{E}_{j}^{\prime},$ set $\mathbb{E}\setminus\{\{\mathbf{N}\}\cup\{-\infty\}\}$ is a communication class, where $\mathbf{N}=(N_{1},\ldots,N_{d})$.

• A3

The matrix $\mathbf{P}_{\hat{X}}$, given below in (12), is non-negative.

Let $X^{*}$ be a chain with the above transition matrix $\mathbf{P}_{X^{*}}$. Assume its initial distribution is $\nu^{*}$. The state $\mathbf{N}$ is absorbing state, denote its absorption time by $T^{*}_{\nu^{*},\mathbf{N}}$.

Then the time to absorption $T^{*}_{\nu^{*},\mathbf{N}}$ has the following pgf

[TABLE]

where $\rho_{j}(1)$ is the winning probability of $X_{j}^{*}$ starting at $1$,

[TABLE]

$\Lambda_{j}$* are given in (21) calculated for $\mathbf{P}_{X^{*}_{j}}^{\prime}$ and $\hat{T}_{\hat{\mathbf{e}},\mathbf{N}}$ is time to absorption in the chain $\hat{X}\sim(\delta_{\hat{\mathbf{e}}},\mathbf{P}_{\hat{X}})$ with:*

$\mathbf{P}_{\hat{X}}((i_{1},\ldots,i_{d}),(i^{\prime}_{1},\ldots,i^{\prime}_{d}))=$**

[TABLE]

We also have

[TABLE]

Moreover, the eigenvalues of $\mathbf{P}_{X^{*}}$ and $\mathbf{P}_{\hat{X}}$ are the diagonal entries of $\mathbf{P}_{\hat{X}}$.

Note that $\hat{X}$ is a pure-birth chain. Moreover, at one step it can change increase coordinates by +1 on a set $B$ such that $B\subseteq A_{k},$ for $k=1,\ldots,m.$

Remark 2.5**.**

In case $b_{i}\geq 0,i=1,\ldots,m$ (i.e., $(b_{1},\ldots,b_{m})$ is a distribution on $\{1,\ldots,m\}$) the matrix $\mathbf{P}_{X^{*}}$ in assumption A2 is stochastic (thus A2 is only about $\mathbb{E}\setminus\{\{\mathbf{N}\}\cup\{-\infty\}\}$ being a communication class) and so is the matrix $\mathbf{P}_{\hat{X}}$ given in (12) (i.e., A3 is fulfilled).

Considering initial distribution having whole mass at $(1,\ldots,1)$ and/or all $q_{j}(1)=0,j=1,\ldots,d$ we have special cases, which we will formulate as a corollary.

Corollary 2.6**.**

Consider setup from Theorem 2.4.

• a)

Moreover, assume that $q_{j}(1)=0$ for all $j=1,\ldots,d$. I.e., each $X^{*}_{j}$ has actually only one absorbing state (state [math] is not accessible). Then, $\mathbf{N}$ is the only absorbing state of $X^{*}$, $\sum_{\mathbf{e}\in\mathbb{E}}\hat{\nu}(\mathbf{e})$=1, $\rho_{j}(1)=1,j=1,\ldots,d$ and we have

[TABLE]

• b)

Moreover, assume that both $q_{j}(1)=0$ for all $j=1,\ldots,d$ and $\nu^{*}((1,\ldots,1))=1$. Then $T^{*}_{(1,\ldots,1),\mathbf{N}}\stackrel{{\scriptstyle d}}{{=}}\hat{T}_{(1,\ldots,1),\mathbf{N}}$, where $\stackrel{{\scriptstyle d}}{{=}}$ denotes equality in distribution.

• c)

Moreover, assume that $\nu^{*}((1,\ldots,1))=1$. Then, assertions of Theorem 2.4 hold with $\hat{\nu}((1,\ldots,1))=1$ and we have

[TABLE]

Sample-path construction

It turns out that when $\hat{\nu}$ resulting from $\hat{\nu}=\nu^{*}\Lambda^{-1}$ is a distribution (which is always the case in, e.g., Corollary 2.6  b) and c), we can have a sample-path construction. I.e., for $X^{*}$ we can construct, sample path by sample path, a chain $\hat{X}$, so that $T^{*}_{\nu^{*},\mathbf{N}}$ has the distribution expressed in terms of $\hat{T}_{\hat{\nu},\mathbf{N}}$ as stated in Theorem 2.4. The construction is analogous to the construction given in [4] (paragraph 2.4) - note however that the construction therein was between ergodic chain and its strong stationary dual chain (i.e., the chain with one absorbing state) and the link $\Lambda$ was a stochastic matrix (it can be substochastic in our case). Having observed Having observed $X^{*}_{0}=\mathbf{e}^{*}_{0}$ (chosen from the distribution $\nu^{*}$) we set

[TABLE]

Then, after choosing $X^{*}_{1}=\mathbf{e}^{*}_{1},\ldots,X^{*}_{n-1}=\mathbf{e}^{*}_{n-1}$ and $\hat{X}_{1}=\hat{\mathbf{e}}_{1},\ldots,\hat{X}_{n}=\hat{\mathbf{e}}_{n}$ we set

[TABLE]

This way we have constructed the chain $\hat{X}$ so that $\Lambda\mathbf{P}_{X^{*}}=\mathbf{P}_{\hat{X}}\Lambda$ and $\nu^{*}=\hat{\nu}\Lambda$ with the property that $\hat{X}_{n}=\hat{\mathbf{e}}_{M}$ if and only if $X^{*}_{n}=\mathbf{e}^{*}_{M}$.

Theorem 2.4 is actually neither an extension of (6) nor (7) to the multidimensional case, since for one-dimensional case the formula for pgf of $T^{*}_{s,N}$ has a different form, as the example given in Section 6.2 shows.

3 Tools: dualities in Markov chains

Siegmund duality and intertwinings between chains are the key ingredients of our main theorems’ proofs.

3.1 Siegmund duality

Let $X$ be an ergodic discrete-time Markov chain with transition matrix $\mathbf{P}_{X}$ and finite state space $\mathbb{E}=\{\mathbf{e}_{1},\ldots,\mathbf{e}_{M}\}$ partially ordered by $\preceq$. Denote its stationary distribution by $\pi$. We assume that there exists a unique minimal state, say $\mathbf{e}_{1}$, and a unique maximal state, say $\mathbf{e}_{M}$. For $A\subseteq\mathbb{E}$, define $\mathbf{P}_{X}(\mathbf{e},A):=\sum_{\mathbf{e}^{\prime}\in A}\mathbf{P}_{X}(\mathbf{e},\mathbf{e}^{\prime})$ and similarly $\pi(A):=\sum_{\mathbf{e}\in A}\pi(\mathbf{e})$. Define also $\{\mathbf{e}\}^{\uparrow}:=\{\mathbf{e}^{\prime}\in\mathbb{E}:\mathbf{e}\preceq\mathbf{e}^{\prime}\}$, $\{\mathbf{e}\}^{\downarrow}:=\{\mathbf{e}^{\prime}\in\mathbb{E}:\mathbf{e}^{\prime}\preceq\mathbf{e}\}$ and $\delta(\mathbf{e},\mathbf{e}^{\prime})=\mathbbm{1}\{\mathbf{e}=\mathbf{e}^{\prime}\}$. We say that a Markov chain $Z$ with transition matrix $\mathbf{P}_{Z}$ is the Siegmund dual of $X$ if

[TABLE]

In all non-degenerated applications, we can find substochastic matrix $\mathbf{P}^{\prime}_{Z}$ fulfilling (14). Then we add one extra state absorbing, say $\mathbf{e}_{0}$, and define $\mathbf{P}_{Z}=\mathcal{F}_{\mathbf{e}_{0}}(\mathbf{P}^{\prime}_{Z})$. Note that then $\mathbf{P}_{Z}$ fulfills (14) for all states different from $\mathbb{E}$. This relation also implies that $\mathbf{e}_{M}$ is an absorbing state in Siegmund dual, thus ${Z}$ has two absorbing states. Taking the limits as $n\to\infty$ on both sides of (14), we have

[TABLE]

The stationary distribution of $X$ is related in this way to the absorption of its Siegmund dual $Z$.

It is convenient to define Siegmund duality in matrix form. Let $\mathbf{C}(\mathbf{e}_{i},\mathbf{e}_{j})=\mathbbm{1}(\mathbf{e}_{i}\preceq\mathbf{e}_{j})$, then the equality (14) can be expressed as

[TABLE]

Relation (15) can be rewritten in matrix form as

[TABLE]

The inverse $\mathbf{C}^{-1}$ always exists, usually is denoted by $\mu$ and called Möbius function. To find a Siegmund dual it is enough to find $\mathbf{P}_{Z}$ fulfilling (16) with for $n=1$.

Let $\preceq:=\leq$ be a total ordering on a finite state space $\mathbb{E}=\{1,\ldots,M\}$. The chain $Y$ is stochastically monotone w.r.t to total ordering if $\forall i_{1}\leq i_{2}\textrm{ }\forall j\textrm{ }\mathbf{P}_{Y}(i_{2},\{j\}^{\downarrow})\leq\mathbf{P}_{Y}(i_{1},\{j\}^{\downarrow})$. We have

Lemma 3.1** (Siegmund [17]).**

Let $X$ be an ergodic Markov chain on $\mathbb{E}=\{1,\ldots,M\}$ with transition matrix $\mathbf{P}_{X}$. Siegmund dual $Z$ (w.r.t. total ordering) exists if and only if $X$ is stochastically monotone. In such a case $\mathbf{P}_{Z}=\mathcal{F}(\mathbf{P}^{\prime}_{Z})$, where

[TABLE]

for $i,j\in\mathbb{E}$ (we mean $\mathbf{P}_{X}(i+1,\cdot)=0$).

Since the proof is one line long, we present it.

Proof 3.2** (Proof of Lemma 3.1).**

The main thing is to show that (14) holds for $n=1$. We have

[TABLE]

The latter is non-negative if and only if $X$ is stochastically monotone.

Let $X$ be and ergodic birth and death chain on $\mathbb{E}=\{1,\ldots,M\}$ with transition matrix

[TABLE]

where $q^{\prime}(1)=p^{\prime}(M)=0$ and $p^{\prime}(i)>0,i=1,\ldots,M-1,q^{\prime}(i)>0,i=2,\ldots,M$. Assume that $p^{\prime}(i-1)+q^{\prime}(i)\leq 1,i=2,\ldots,M$ (what is equivalent to stochastic monotonicity).

It is easily verifiable that when we rename transition probabilities: $p(i)=q^{\prime}(i),q(i)=p^{\prime}(i-1)$, then the transitions $\mathbf{P}_{Y}$ defined in (4) are the transitions of Siegmund dual resulting from Lemma 3.1. From the known form of stationary distribution of an ergodic birth and death chain, and from relation (15), it follows that for $\mathbf{P}_{Y}$ given in (4) we have

[TABLE]

3.2 Intertwinings between absorbing chains

Let $\Lambda$ be any nonsingular matrix of size $M\times M$. We say that matrices $\mathbf{P}_{X^{*}}$ and $\mathbf{P}_{\hat{X}}$ of size $M\times M$ are intertwined by a link $\Lambda$ if

[TABLE]

Similarly, we say that vectors $\hat{\nu}$ and $\nu^{*}$ of length $M$ are intertwined if

[TABLE]

We say that link $\Lambda$ is $\mathbf{e}_{M}^{*}$-isolated if

[TABLE]

Lemma 3.3**.**

Let $X^{*}$ and $\hat{X}$ be Markov chains on $\mathbb{E}^{*}=\mathbf{e}_{0}^{*}\cup\hat{\mathbb{E}}$ and $\hat{\mathbb{E}}$ with transition matrices $\mathbf{P}_{X^{*}}$ and $\mathbf{P}_{\hat{X}}$ respectively. Moreover, assume $X^{*}$ has initial distribution $\nu^{*}$ and two absorbing states : $\mathbf{e}_{0}^{*}$ and $\mathbf{e}_{M}^{*}$, whereas $\hat{X}$ has one absorbing state $\hat{\mathbf{e}}_{M}$. Assume that $\mathbf{P}_{X^{*}}^{\prime}=\mathcal{F}^{-1}_{\mathbf{e}_{0}^{*}}(\mathbf{P}_{X^{*}})$ and $\mathbf{P}_{\hat{X}}$ are intertwined via $\mathbf{e}_{M}^{*}$-isolated link $\Lambda$. Let $\hat{\nu}=\nu^{*}\Lambda^{-1}$. Then we have

[TABLE]

Proof 3.4**.**

[TABLE]

Now, for $pgf$ we have:

$pgf_{T^{*}_{\nu^{*},\mathbf{e}^{*}_{M}}}(s)=$**

[TABLE]

Corollary 3.5**.**

Let assumptions of Lemma 3.3 hold and, in addition, let $\hat{\nu}$ be a distribution. Then, we have

[TABLE]

From Fill, Lyzinski [6] we can deduce the following.

Lemma 3.6**.**

Let $X^{*}$ be a birth and death chain on $\mathbb{E}^{*}=\{0,\ldots,M\}$ with transition matrix $\mathbf{P}_{X^{*}}$ given in (4) with two absorbing states: [math] and $M$. Let $\mathbf{P}_{X^{*}}^{\prime}=\mathcal{F}^{-1}_{0}(\mathbf{P}_{X^{*}})$. Assume the eigenvalues of $\mathbf{P}_{X^{*}}^{\prime}$ are non-negative, denote them by $0\leq\lambda_{1}\leq\ldots\leq\lambda_{M}=1$.

Define $\mathbf{Q}_{1}:=\mathbf{I}$ and

[TABLE]

Let ${\Lambda}$ be the lower triangular square matrix of size $M\times M$ defined as

[TABLE]

Then, $\mathbf{P}_{X^{*}}^{\prime}$ and $\mathbf{P}_{\hat{X}}$ are intertwined via link $\Lambda,$ where

[TABLE]

is a matrix of size $M\times M$.

Note that Lemma 3.6 is similar to Theorem 4.2 in [6], the difference is that therein $\Lambda$ is a stochastic matrix, whereas in Lemma 3.6 it can be substochastic (it is strictly substochastic if $q(0)>0$). Almost identical $\Lambda$ was considered in [7], their Proposition 3.3 yields.

Lemma 3.7**.**

• •

The matrices $\mathbf{Q}_{k},1,\ldots,M$ are non-negative and substochastic.

• •

The matrix $\Lambda$ is non-negative and substochastic, it is lower triangular and

[TABLE]

thus $\Lambda$ is nonsingular.

Remark 3.8**.**

Note that in case $X^{*}$ has no transition to 0, i.e., $q(1)=0$, it is actually a chain on $\{1,\ldots,M\}$ and $\mathbf{P}_{X^{*}}^{\prime}=\mathcal{F}^{-1}_{0}(\mathbf{P}_{X^{*}})$ is a stochastic matrix. Then $\Lambda$ is a stochastic matrix and $\Lambda(M,M)=1$.

4 Proofs

4.1 Properties of Kronecker product

In this section we recall some useful properties of Kronecker product and formulate lemma relating eigenvectors and eigenvalues of some combination of Kronecker products.

We will exploit the following properties

• •

bilinearity:

[TABLE]

• •

mixed product:

[TABLE]

• •

inverse and transposition:

[TABLE]

[TABLE]

• •

eigenvalue and eigenvector:

[TABLE]

Last property leads us to the following lemma.

Lemma 4.1**.**

*For all $1\leq j\leq n$ and $1\leq i\leq m$, let $a_{j}$ be the left eigenvectors with the corresponding eigenvalues $\alpha_{j}$ of square matrices $\mathbf{A}_{j}^{(i)}$ of size $k_{j}$ respectively.

Let $\mathbf{B}_{i},i=1,\ldots,m$ be square matrices of size $\prod_{j=1}^{n}k_{j}$ such that $\sum_{i=1}^{m}\mathbf{B}_{i}=\mathbf{I}$, where $\mathbf{I}$ is identity of size $\prod_{j=1}^{n}k_{j}$. Then $\prod_{j\leq n}\alpha_{j}$ with $\bigotimes_{j\leq n}a_{j}$ are the eigenvalue and the left eigenvector of $\mathbf{A}=\sum_{i=1}^{m}(\bigotimes_{j\leq n}\mathbf{A}_{j}^{(i)})\mathbf{B}_{i}$.

Similarly, if $b_{i},i=1,\ldots,m$ are real numbers such that $\sum_{i=1}^{m}b_{i}$ we have that $\bigotimes_{j\leq n}a_{j}$ is the left eigenvector with the corresponding eigenvalue $\prod_{j\leq n}\alpha_{j}$ of the matrix $\mathbf{A}=\sum_{i=1}^{m}(\bigotimes_{j\leq n}\mathbf{A}_{j}^{(i)})b_{i}$.*

Proof 4.2**.**

We have

[TABLE]

Similarly,

[TABLE]

Substitution of stochastic matrices $\mathbf{P}_{j}^{(i)}$ with stationary distributions $\pi_{j}$ (for all $1\leq j\leq n$, $1\leq i\leq m$) to matrices $\mathbf{A}_{j}^{(i)}$ with left eigenvectors $\alpha_{j}$ (for all $1\leq j\leq n$, $1\leq i\leq m$) gives us following Corollary (keeping in mind that 1 is the eigenvalue corresponding to the eigenvector being the stationary distribution):

Corollary 4.3**.**

Let $\mathbf{P}_{j}^{(i)}$ be a stochastic matrix of size $k_{j}$ with $\pi_{j}$ the stationary distribution for all $1\leq j\leq n,1\leq i\leq m$. Let $\mathbf{B}_{i}$, $1\leq i\leq m$ be square matrices of size $\prod_{j=1}^{n}k_{j}$ such that $\sum_{i=1}^{m}\mathbf{B}_{i}=\mathbf{I}$, where $\mathbf{I}$ is the identity of size $\prod_{j=1}^{n}k_{j}$. Similarly, if $b_{i}$, $1\leq i\leq m$ are real numbers such that $\sum_{i=1}^{m}b_{i}=1$, then the stochastic matrices of the form $\sum_{i=1}^{m}(\bigotimes_{j\leq n}\mathbf{P}_{j}^{(i)})\mathbf{B}_{i}$ or $\sum_{i=1}^{m}(\bigotimes_{j\leq n}\mathbf{P}_{j}^{(i)})b_{i}$ have stationary distribution of the form $\bigotimes_{j\leq n}\pi_{j}$.

4.2 Proof of Theorem 2.1

We will find an ergodic Markov chain $X$ with transition matrix $\mathbf{P}_{X}$ and some partial ordering of the state space (expressed by an ordering matrix $\mathbf{C}$) and show that (15) is equivalent to (9).

Let $\mathbf{P}_{Z_{j}}^{(k)}$ (on $\mathbb{E}_{j}=\{0,\ldots,N_{j}\}$) be as in theorem. Let $X_{j}^{(k)}$ be ergodic chains on $\mathbb{E}^{\prime}_{j}=\{1,\ldots,N_{j}\}$ with transition matrix $\mathbf{P}_{X_{j}}^{(k)}$, such that $Z_{j}^{(k)}$ is its Siegmund dual w.r.t. total ordering. I.e., let $\mathbf{C}_{j}(s,t)=\mathbbm{1}(s\leq t)$, and duality means that

[TABLE]

where $\mathbf{P}_{Z_{j}^{\prime}}^{(k)}=\mathcal{F}^{-1}_{0}(\mathbf{P}_{Z_{j}}^{(k)})$. Assumption (8) and relation (15) imply that for fixed $j$, the chains $X_{j}^{(k)},k=1,\ldots,m$ have the same stationary distribution, denote it by $\pi_{j}$. The relation (15) means that $\boldsymbol{\rho}_{j}=\boldsymbol{\pi}_{j}\mathbf{C}_{j}$. On the state space $\mathbb{E}=\bigotimes_{j\leq d}\mathbb{E}_{j}$ let us introduce the ordering expressed by matrix $\mathbf{C}=\bigotimes_{j\leq d}\mathbf{C}_{j}$. From (16) we can calculate the matrix $\mathbf{P}_{X}$:

[TABLE]

Let us define

[TABLE]

In the case $j\in A_{k}$, the distribution $\pi_{j}$ is the unique stationary distribution. In the case $j\notin A_{k}$, any distribution is an invariant measure, however, we fix it to be $\pi_{j}$.

We have

[TABLE]

From property P1 we have that

[TABLE]

thus Corollary 4.3 implies that $\boldsymbol{\pi}=\bigotimes_{j\leq d}\boldsymbol{\pi}_{j}$ is the stationary distribution of $\mathbf{P}_{X}$, thus $\boldsymbol{\rho}=\boldsymbol{\pi}\mathbf{C},$ what is equivalent to (9).

$\square$

4.3 Proof of Theorem 2.4

To prove the theorem we will construct an $\mathbf{N}$-isolated link $\Lambda$, so that $\mathbf{P}_{X^{*}}^{\prime}$ and $\mathbf{P}_{\hat{X}}$, given in (11) and (12) respectively, are intertwined via this link.

Consider matrix $\mathbf{P}_{X^{*}_{j}}^{\prime}$. Define stochastic matrix $\mathbf{P}_{\hat{X}_{j}}$ of size $N_{j}\times N_{j}$ defined as:

[TABLE]

Let $\Lambda_{j}$ be a link intertwining matrices $\mathbf{P}_{X^{*}_{j}}^{\prime}$ and ${\mathbf{P}}_{\hat{X}_{j}}$ given in (21). Define

[TABLE]

Note that matrices $\mathbf{R}_{X^{*}_{j}}^{\prime}$ and $\mathbf{R}_{\hat{X}_{j}^{(k)}}$ are also intertwined via $\Lambda_{j}$ for any $j=1,\ldots,d$ and any $k=1,\ldots,m$. Any link intertwines two identity matrices, which is the case for $j\notin A$. I.e., we have $\Lambda_{j}\mathbf{R}_{X^{*}_{j}}^{\prime}=\mathbf{R}_{\hat{X}_{j}^{(k)}}\Lambda_{j},j=1,\ldots,d$. Define

[TABLE]

We have

[TABLE]

Simple calculations yield that $\mathbf{P}_{\hat{X}}$ given in (12) can be written as $\displaystyle\sum_{k=1}^{m}b_{k}\left(\bigotimes_{j\leq d}\mathbf{R}_{\hat{X}_{j}^{(k)}}\right)$. Thus, we have $\Lambda\mathbf{P}_{X^{*}}^{\prime}=\mathbf{P}_{\hat{X}}\Lambda$. Now, let us calculate $\hat{\nu}=\nu^{*}\Lambda^{-1}$ (note that $\Lambda$ is nonsingular because of property (P3) and fact that each $\Lambda_{j},j=1,\ldots,d$ and identity matrices $\mathbf{I}_{j}$ are nonsingular). In other words, we have $\nu^{*}=\hat{\nu}\Lambda$. Equation (13) holds, since $\Lambda$ is lower triangular.

Moreover, $\Lambda$ is $(N_{1},\ldots,N_{d})$-isolated, since we have

[TABLE]

where in $\stackrel{{\scriptstyle(*)}}{{=}}$ we used Lemma 3.7. Applying Lemma 3.3 completes the proof.

$\square$

5 Outline of alternative proof of Theorem 2.4: strong stationary duality approach

In Theorem 2.4 we related absorption time $T^{*}_{\nu^{*},\mathbf{N}}$ of $X^{*}$ with absorption time $\hat{T}_{\hat{\nu},\mathbf{N}}$ of $\hat{X}$. This was done by finding a specific matrix $\Lambda$, such that $\Lambda\mathbf{P}_{X^{*}}=\mathbf{P}_{\hat{X}}\Lambda$, exploiting existence of such $\Lambda$ for $X^{*}$ and $\hat{X}$ being birth and death chains. The exploited $\Lambda$ is related to spectral polynomials of the stochastic matrix $\mathbf{P}_{X^{*}}$. Such link appeared first naturally as a link between an ergodic chain $X$ and an absorbing chain $X^{*}$. The proof of Theorem 2.4 in case $q_{j}(1)=0$ (i.e., Corollary 2.6 a)) can be different, using intermediate ergodic chain. In this section we will describe its outline.

Strong stationary duality

Let $X$ be an ergodic Markov chain on $\mathbb{E}=\{\mathbf{e}_{1},\ldots,\mathbf{e}_{N}\}$ with initial distribution $\nu$ and transition matrix $\mathbf{P}_{X}$. Let ${\mathbb{E}}^{*}=\{{\mathbf{e}_{1}^{*}},\ldots,{\mathbf{e}_{M}^{*}}\}$ be the, possibly different, state space of the absorbing Markov chain $X^{*}$, with transition matrix ${\mathbf{P}_{X^{*}}}$ and initial distribution ${\nu^{*}}$, whose unique absorbing state is denoted by ${\mathbf{e}_{M}^{*}}$. Assume that $\Lambda^{*}$ is a stochastic $M\times N$ matrix satisfying $\Lambda({\mathbf{e}_{M}^{*}},\mathbf{e})=\pi(\mathbf{e})$. We say that $X^{*}$ is a strong stationary dual (SSD) of $X$ with link $\Lambda^{*}$ if

[TABLE]

Diaconis and Fill [4] prove that then the absorption time ${T^{*}}$ of $X^{*}$ is the so called strong stationary time for $X$. This is a random variable $T$ such that $X_{T}$ has distribution $\pi$ and $T$ is independent of $X_{T}$. The main application is to studying the rate of convergence of an ergodic chain to its stationary distribution, since for such a random variable we always have $d_{TV}(\nu\mathbf{P}_{X}^{k},\pi)\leq sep(\nu\mathbf{P}_{X}^{k},\pi)\leq P(T>k)$, where $d_{TV}$ stands for the total variation distance, and $sep$ stands for the separation ‘distance’. For details, see Diaconis and Fill [4]. We say that SSD is sharp if ${T^{*}}$ corresponds to stochastically the smallest SST, then we have $sep(\nu\mathbf{P}_{X}^{k},\pi)=P({T^{*}}>k)$, the corresponding SST $T^{*}$ is often called fastest strong stationary time (FSST).

Strong stationary duality for birth and death chain

Let $X$ be an ergodic birth and death chain on $\mathbb{E}=\{1,\ldots,M\}$, whose time reversal is stochastically monotone. With transitions given in (17). Diaconis and Fill [4] show that an absorbing birth and death chain $X^{*}$ on $\mathbb{E}^{*}=\mathbb{E}$ with transitions given by

[TABLE]

is a sharp SSD for $X$. Here we have

[TABLE]

Moreover, starting from an absorbing birth and death chain $X^{*}$ on $\mathbb{E}=\{1,\ldots,M\}$, whose unique absorbing state is $M$, Theorem 3.1 in [5] states that we can find an ergodic chain $X$ (and some stationary distribution $\pi$), such that $X^{*}$ is its sharp SSD with link given in (24).

Spectral pure-birth chain

Again, let $X$ be an ergodic birth and death chain on $\mathbb{E}=\{1,\ldots,M\}$. Assume its eigenvalues are non-negative, $0\leq\lambda_{1}\leq\ldots\leq\lambda_{M}=1$. Then, the chain $\hat{X}$ with transitions given in (22) is its sharp SSD with link $\hat{\Lambda}$ given in (21).

The outline of an alternative proof

As in Section 4.3, the main idea is to show that two absorbing birth and death chains $X^{*}_{j}$ and $\hat{X}_{j}$ (pure-birth) on $\mathbb{E}_{j}=\{1,\ldots,N_{j}\}$ are intertwined by an $N_{j}$-isolated link $\Lambda_{j}$. Collecting above findings, we have (skipping conditions on initial distributions):

• •

Let $X_{j}$ be an ergodic chain on $\mathbb{E}_{j}$, whose $X^{*}_{j}$ is a sharp SSD, i.e., we have $\Lambda_{j}^{*}\mathbf{P}_{X_{j}}=\mathbf{P}_{X_{j}^{*}}\Lambda_{j}^{*}$.

• •

Let $\hat{X}_{j}$ be a pure-birth sharp SSD for $X_{j}$, i.e., we have $\hat{\Lambda}_{j}\mathbf{P}_{X}=\mathbf{P}_{\hat{X}_{j}}\hat{\Lambda}_{j}$.

It means that absorption times $T^{*}$ and $\hat{T}$ are equal in distribution (since both $X^{*}$ and $\hat{X}$ are sharp SSDs of $X$. Mathematically, we have

[TABLE]

i.e., ${X^{*}_{j}}$ and $\hat{X}_{j}$ are intertwined by the link $\Lambda_{j}$, which is $N_{j}$-isolated. Intertwining between two absorbing birth and death chains via an ergodic chain is depicted in Fig. 1. Taking $\Lambda=\bigotimes_{j\leq d}\Lambda_{j}$ and $\hat{\nu}=\nu^{*}\Lambda^{-1}$ we proceed with the proof of Theorem 2.4 as in Section 4.3.

6 Examples

6.1 One dimensional gambler’s ruin problem with $N=3$: calculating $T^{*}_{2,3}$

Here we present a simple example for calculating $T^{*}_{2,3}$ in a one dimensional gambler’s ruin problem using Theorem 2.4. We also check that calculations agree with formula (6).

Example 6.1**.**

Let $d=1,N_{1}=3$ and $p_{1}(1)=p_{1}(2)=p>0,q_{1}(1)=q_{1}(2)=q>0$ such that $p\neq q$ and $p+q+\sqrt{pq}<1$. The transition matrix of $\mathbf{P}_{X^{*}_{1}}$ is following

[TABLE]

Then, the $pgf$ of time to absorption starting at 2 is given by:

[TABLE]

Proof 6.2**.**

We have $\mathbf{P}_{X^{*}}=\mathbf{P}_{X^{*}_{1}}$. The eigenvalues of $\mathbf{P}_{X^{*}}^{\prime}=\mathcal{F}_{0}^{-1}(\mathbf{P}_{X^{*}})$ are $\lambda_{1}=1-p-q-\sqrt{pq},\lambda_{2}=1-p-q+\sqrt{pq},\lambda_{3}=1$. The transitions of $\mathbf{P}_{\hat{X}}$ are following

[TABLE]

Thus,

[TABLE]

Calculating $\Lambda$ from (21) (for $\mathbf{P}_{X^{*}}$) we obtain

[TABLE]

Calculations yield (we have $\nu^{*}(2)=1$)

[TABLE]

From (6) we have $\rho(i)={1-\left({q\over p}\right)^{i}\over 1-\left({q\over p}\right)^{3}},i=1,2,3$. Finally,

[TABLE]

what can be written as (25). On the other hand, (7) states that

[TABLE]

where $\lambda_{1}^{\lfloor 2\rfloor}=1-(p+q)$, which, as can be checked, is equivalent to (25).

6.2 Winning probabilities and absorption time: changing $r$ coordinates at one step in $d$-dimensional game

We will present an example for both Theorems, 2.1 and 2.4. The chains $\mathbf{P}_{Z^{(k)}_{j}}$ in Theorem 2.1 are quite general, but in this example we consider birth and death chains i.e., we will use $\mathbf{P}_{Z^{(k)}_{j}}=\mathbf{P}_{X^{*}_{j}}$ from Theorem 2.4 (birth and death chains given in (4)). Similarly, we have $\mathbf{R}_{Z^{(k)}_{j}}^{\prime}=\mathbf{R}_{X^{*(k)}_{j}}^{\prime}$ and $\mathbf{P}_{Z}=\mathbf{P}_{X^{*}}$.

Example 6.3**.**

The idea of the example is the following. We construct $d$-dimensional game from one-dimensional games, in such a way, that at one step we play with $r$ other players, where $r\in\{1,\ldots,d\}$. In other words, the multidimensional chain can change at most $r$ coordinates in one step.

Moreover we will take, as $\mathbf{B}_{i}:=b_{i}$ real numbers. In both theorems let us take $0<r<d$, $m={{d}\choose{r}}+1$ and $b_{k}=1,k=1,\ldots,m-1,b_{m}=1-{{d}\choose{r}}.$ Let us enumerate combinations of $r$ positive numbers no greater than $d$ in some way (see e.g.,[14]). Let $\mathbf{A}_{k}$ be k-th combination from this numbering, for $k=1,\ldots,m-1$ and $\mathbf{A}_{m}=\emptyset$. Then we have

[TABLE]

We have that $\mathbf{R}^{\prime}_{Z_{j}^{(k)}}=\mathbf{P}^{\prime}_{Z_{j}}$ if $\{j\}\in\mathbf{A}_{k}$ and $\mathbf{R}^{\prime}_{Z_{j}^{(k)}}=\mathbf{I}_{j}$ otherwise (for $\{j\}\not\in\mathbf{A}_{k}$), thus $\mathbf{P}^{\prime}_{Z}=$

[TABLE]

In other words, we combine $d$ one-dimensional birth and death chains in such a way, that the resulting $d$-dimensional chain can change at most $r$ coordinates by $\pm 1$ at one step.

We can rewrite this formula for some cases:

• •

$r=d$, independent games

[TABLE]

• •

$r=d-1$

[TABLE]

• •

$r=2$

[TABLE]

• •

$r=1$

[TABLE]

This can be rewritten as

[TABLE]

$\mathbf{P}_{Z}=\mathcal{F}_{-\infty}(\mathbf{P}^{\prime}_{Z})$ are exactly the transition corresponding to the generalized gambler’s ruin problem given in (1).

In all above cases, the winning probability for chain $\mathbf{P}_{Z}$ is given in (3). This is since the winning probabilities for $\mathbf{P}_{Z_{j}}$ are given in (10), thus using (9) the relation is (3) proven.

In all above cases, if we replace $\mathbf{P}_{Z_{j}}^{\prime}$ with $\mathbf{P}_{\hat{X}_{j}}$ and $\mathbf{P}_{Z}^{\prime}$ with $\mathbf{P}_{\hat{X}}$, then we have a special cases for formula for $\mathbf{P}_{\hat{X}}$ given in (12). If, in addition, we assume that $\nu^{*}((1,\ldots,1))$, then, from Corollary 2.6 c) we have

[TABLE]

For example, in case $r=1$ (then we have $m=d+1$ and take $b_{k}=1,k=1,\ldots,d,b_{d+1}=1-d,\mathbf{A}_{k}=\{k\},k=1,\ldots,d$ and $\mathbf{A}_{d+1}=\emptyset$) we have

[TABLE]

Sample transitions for case $d=2,r=1$ are depicted in Fig. 2.

In Figure 3 the transition of $\hat{X}$ are presented for $d=3$:

• •

When $r=1$ only blue are possible.

• •

When $r=2$ only blue and green are possible.

• •

When $r=3$ all transitions, blue, green and red are possible.

6.3 One dimensional gambler’s ruin problem related to Ehrenfest model: calculating $T^{*}_{m,N}$

Here we present a concrete example of a birth and death chain on $\mathbb{E}=\{1,\ldots,N\}$ with $N$ being the only absorbing state, for which we provide $pgf$ of absorption time provided chain started at any $m\in\mathbb{E}$. We use Lemma 3.3 calculating link $\Lambda$. As far as we are aware, this $pgf$ cannot be given using results from [7] i.e., (6). This is since the eigenvalues the presented matrix $\mathbf{P}_{X^{*}}$ are known, but the eigenvalues of $\mathbf{P}_{X^{*}}^{\lceil m\rceil}$ are not known for any $m\in\mathbb{E}$.

Example 6.4**.**

Let $X^{*}$ be a Markov Chain on the state space $\mathbb{E}=\{1,\ldots,N\}$ with transition matrix ${\mathbf{P}}_{X^{*}}$ of the form:

[TABLE]

Then the absorption time starting at $m\in\mathbb{E}$ is has the following $pgf$:

[TABLE]

where

[TABLE]

In particular, we have

[TABLE]

Proof 6.5**.**

Let

[TABLE]

To show the result via Lemma 3.3 it is enough to find $\Lambda$ such that $\Lambda\mathbf{P}_{X^{*}}=\mathbf{P}_{\hat{X}}\Lambda$ and $\nu^{*}\Lambda^{-1}=\hat{\nu},$ where $\nu^{*}(j)=\mathbbm{1}\{j=m\}$.

However, since $X^{*}$ has only one absorbing state, we can – and we will – follow the outline of an alternative proof given in Section 5. I.e., we will indicate intermediate ergodic chain $X$ on $\mathbb{E}$ with transition matrix $\mathbf{P}_{X}$ and find $\Lambda^{*}$ and $\hat{\Lambda}$ such that $\Lambda^{*}\mathbf{P}_{X}=\mathbf{P}_{X^{*}}\Lambda^{*}$ and $\hat{\Lambda}\mathbf{P}_{X}=\mathbf{P}_{\hat{X}}\hat{\Lambda}$. Then, we will automatically have $\Lambda=\hat{\Lambda}(\Lambda^{*})^{-1}$ and we will show that $\nu^{*}\Lambda^{-1}=\nu^{*}\Lambda^{*}\hat{\Lambda}^{-1}=\hat{\nu}$.

Let $X$ be a chain on $\mathbb{E}$ with the following transition matrix:

[TABLE]

i.e.,* $X$ corresponds to Ehrenfest model of $N-1$ particles with an extra probability (half) of staying (and states are enumerated $1,\ldots,N$, whereas in the classical Ehrenfest model these are $0,\ldots,N-1$). Its stationary distribution is a binomial distribution $\pi(j)={1\over 2^{N-1}}{N-1\choose j-1}$, thus the classical link (cf. (24)) is given by*

[TABLE]

i.e.,* we have $\Lambda^{*}\mathbf{P}_{X}=\mathbf{P}_{X^{*}}\Lambda$ ($X^{*}$ is a sharp SSD of $X$). The eigenvalues of $X$ are known, these are ${i\over N-1},i=0,\ldots,N-1$, thus $\hat{X}$ is its pure birth spectral dual. The link $\hat{\Lambda}$ such that $\hat{\Lambda}\mathbf{P}_{X}=\mathbf{P}_{\hat{X}}\hat{\Lambda}$ is known (see Eq. (4.6) in [5]), it is given by*

[TABLE]

It can be checked that

[TABLE]

Note that $i$-th row of $\hat{\Lambda}^{-1}$ corresponds to the coefficients222The on-line encyclopedia of integer sequences. Sequence A303872.* in expansion of $(2x-1)^{i-1}$.*

Thus, as outlined in Section 5 we have $\Lambda\mathbf{P}_{X^{*}}=\mathbf{P}_{\hat{X}}\Lambda$ with $\Lambda=\hat{\Lambda}(\Lambda^{*})^{-1}$. We need only to check that $\nu^{*}\Lambda^{-1}=\nu^{*}{\Lambda^{*}}\hat{\Lambda}^{-1}$ is equal to $\hat{\nu}$ given in (27). We have

[TABLE]

where in $\stackrel{{\scriptstyle(*)}}{{=}}$ we used identity ${N-1\choose k-1}{k-1\choose j-1}={N-1\choose j-1}{N-j\choose N-k}$. As for the last sum we have

[TABLE]

where in $\stackrel{{\scriptstyle(**)}}{{=}}$ we used identity333See Partial sums at https://en.wikipedia.org/wiki/Binomial_coefficient* $\sum_{k=0}^{M}(-1)^{k}{n\choose k}=(-1)^{M}{n-1\choose M}$. Finally,*

[TABLE]

what is equal to (27).

Note that $pgf$ given in (27) corresponds to the distribution of $\sum_{k=j}^{N-1}Y_{k}$, where $Y_{k}$ is a geometric random variable with parameter ${k-1\over N-1}$ and $Y_{1},\ldots,Y_{N-1}$ are independent. We have $EY_{k}={N-1\over N-k}$ thus (28) follows from (26) and (27).

Remark 6.6**.**

Calculating $\hat{\nu}$ we have actually calculated the link $\Lambda$, which is given by

[TABLE]

\ACKNO

Authors thank Bartłomiej Błaszczyszyn for helpful discussions, in particular for suggesting exploiting Kronecker products.