Normal distributions of finite Markov chains

TL;DR

This paper presents a novel way to express the stationary distribution of finite Markov chains as sums of specific normal distributions linked to planar graphs with loops, building on previous algebraic methods.

Contribution

It introduces a new representation of stationary distributions using normal distributions associated with particular planar graphs, extending prior algebraic approaches.

Findings

Stationary distributions can be expressed as sums of normal distributions.

Normal distributions are associated with planar graphs with loops.

The approach builds on semaphore codes and graph expansions.

Abstract

We show that the stationary distribution of a finite Markov chain can be expressed as the sum of certain normal distributions. These normal distributions are associated to planar graphs consisting of a straight line with attached loops. The loops touch only at one vertex either of the straight line or of another attached loop. Our analysis is based on our previous work, which derives the stationary distribution of a finite Markov chain using semaphore codes on the Karnofsky--Rhodes and McCammond expansion of the right Cayley graph of the finite semigroup underlying the Markov chain.

Equations

$$\text{mathbbm}1\stackrel{a_1}{⟶}{v}_1\stackrel{a_2}{⟶}\cdots \stackrel{a_k}{⟶}{v}_k=n,$$

$$L^{\star }=\bigcup _{i⩾0}{L}^i.$$

$$\{{\ell }_1,{\ell }_2,\dots ,{\ell }_k{\}}^{\star }$$

$$\{a{\}}^{\star }=a^{\star }\text{and}\{a,b{\}}^{\star }=(a^{\star }b{)}^{\star }{a}^{\star }\text{for a,b∈A.}$$

$$L=a{\ell }_1^{\star }bcx,$$

$${\ell }_1=b\{{\ell }_1^{\prime },{\ell }_2^{\prime }{\}}^{\star }da,$$

$$L=a(b\{a,c{\}}^{\star }da{)}^{\star }bcx=a(b(a^{\star }c{)}^{\star }{a}^{\star }da{)}^{\star }bcx,$$

$${\Psi }_G=\sum _{p\in {\mathcal{P}}_G}\prod _{a\in p}{x}_a.$$

$$\sum _{s\in a^{\star }}\prod _{i\in s}{x}_i=\sum _{\ell =0}^{\infty }{x}_a^{\ell }=\frac{1}{1-x_a}.$$

$$\sum _{s\in \{a,b{\}}^{\star }}\prod _{i\in s}{x}_i=\sum _{s\in a^{\star }(b{a}^{\star }{)}^{\star }}\prod _{i\in s}{x}_i=\frac{1}{1-x_a}\cdot \frac{1}{1-\frac{x_b}{1-x_a}}=\frac{1}{1-x_a-x_b}.$$

$$\sum _{s\in \{a_1,a_2,\dots ,a_n{\}}^{\star }}\prod _{i\in s}{x}_i=\frac{1}{1-x_{a_1}-x_{a_2}-\cdots -x_{a_n}}.$$

$${\Omega }_i\cap {\Omega }_j=\varnothing \text{for i=j and}\Omega =\bigcup _{i=1}^{\ell }{\Omega }_i.$$

$$\sum _{t\in {\Omega }_j}{\mathcal{T}}_{t,s}=\sum _{t\in {\Omega }_j}{\mathcal{T}}_{t,s^{\prime }}\text{for all s,s′∈Ωi.}$$

$${\mathcal{T}}_{s^{\prime },s}=\sum _{\begin{array}{c}a\in A\\ s\stackrel{a}{⟶}{s}^{\prime }\end{array}}{x}_a\text{for s,s′∈Ω.}$$

$$p=\left(v_1\stackrel{a_1}{⟶}\cdots \stackrel{a_{\ell }}{⟶}{v}_{\ell +1}\right),$$

$$p:=\left(\text{mathbbm}1\stackrel{a_1}{⟶}{v}_1\stackrel{a_2}{⟶}\cdots \stackrel{a_{\ell }}{⟶}{v}_{\ell }\right)\text{and}{p}^{\prime }:=\left(\text{mathbbm}1\stackrel{a_1^{\prime }}{⟶}{v}_1^{\prime }\stackrel{a_2^{\prime }}{⟶}\cdots \stackrel{a_{{\ell }^{\prime }}^{\prime }}{⟶}{v}_{{\ell }^{\prime }}^{\prime }\right)$$

$$[p{]}_S:=\left(\text{mathbbm}1\stackrel{a_1}{⟶}[v_1{]}_S\stackrel{a_2}{⟶}\cdots \stackrel{a_{\ell }}{⟶}[v_{\ell }{]}_S\right)\text{and}[p^{\prime }{]}_S:=\left(\text{mathbbm}1\stackrel{a_1^{\prime }}{⟶}[v_1^{\prime }{]}_S\stackrel{a_2^{\prime }}{⟶}\cdots \stackrel{a_{{\ell }^{\prime }}^{\prime }}{⟶}[v_{{\ell }^{\prime }}^{\prime }{]}_S\right),$$

$${\Psi }_w^{\mathcal{M}(S,A)}=\sum _{\begin{array}{c}v\in \mathsf{KR}(S,A)\\ [v{]}_S=w\end{array}}{\Psi }_v^{\mathcal{M}(\mathsf{KR}(S,A))}\text{for all w∈(S,A).}$$

$${\Psi }_w^{\mathcal{M}(\mathsf{KR}(S,A))}=\sum _{\begin{array}{c}s\in \mathcal{S}(S,A)\\ [s{]}_{\mathsf{KR}(S,A)}=w\end{array}}\prod _{a\in s}{x}_a\text{for all w∈K(KR(S,A)).}$$

$${\Psi }_w^{\mathcal{M}(\mathsf{KR}(S,A))}=\underset{x_{\square }\to 0}{\lim }{\Psi }_w^{\mathcal{M}(\mathsf{KR}(S\cup \{\square \},A\cup \{\square \}))}.$$

$$\begin{array}{rl}E:=\{(p,a,q)\in V\times A\times V\mid & \tau (q)=\tau (p)a,\ell (q)⩽\ell (p)+1,\\ & \text{q is an initial segment of p if ℓ(q)⩽ℓ(p)}\}.\end{array}$$

$$p=\left(\text{mathbbm}1\stackrel{a_1}{⟶}{v}_1\stackrel{a_2}{⟶}\cdots \stackrel{a_{\ell }}{⟶}{v}_{\ell }\right)$$

$$L=a\{{\ell }_1,{\ell }_2,{\ell }_3,{\ell }_4{\}}^{\star }b{\ell }_5^{\star }\square ,$$

$$\begin{array}{rl}{\ell }_1& =a(b(aa{)}^{\star }b{)}^{\star }b(aa{)}^{\star }ab,\\ {\ell }_2& =a(b(aa{)}^{\star }b{)}^{\star }a,\\ {\ell }_3& =b(a(bb{)}^{\star }a{)}^{\star }a(bb{)}^{\star }ba,\\ {\ell }_4& =b(a(bb{)}^{\star }a{)}^{\star }b,\\ {\ell }_5& =a(bb{)}^{\star }a.\end{array}$$

$$\begin{array}{rl}{\Psi }_{\mathsf{Pict}(\Gamma ,ab\square )}& =\frac{x_a{x}_b{x}_{\square }}{\left(1-\frac{x_a^2{x}_b^2}{\left(1-\frac{x_b^2}{1-x_a^2}\right)(1-x_a^2)}-\frac{x_a^2}{1-\frac{x_b^2}{1-x_a^2}}-\frac{x_a^2{x}_b^2}{\left(1-\frac{x_a^2}{1-x_b^2}\right)(1-x_b^2)}-\frac{x_b^2}{1-\frac{x_a^2}{1-x_b^2}}\right)\left(1-\frac{x_a^2}{1-x_b^2}\right)}\\ & =\frac{x_a{x}_b{x}_{\square }(1-x_b^2)}{\left(1-\frac{2x_a^2{x}_b^2}{1-x_a^2-x_b^2}-\frac{x_a^2(1-x_a^2)}{1-x_a^2-x_b^2}-\frac{x_b^2(1-x_b^2)}{1-x_a^2-x_b^2}\right)(1-x_a^2-x_b^2)}\\ & =\frac{x_a{x}_b{x}_{\square }(1-x_b^2)}{1-2x_a^2-2x_b^2+(x_a^2-x_b^2{)}^2}.\end{array}$$

$$\underset{x_{\square }\to 0}{\lim }{\Psi }_{\mathsf{Pict}(\Gamma ,ab\square )}=\frac{1}{8}(1-x_b^2).$$

$$\begin{array}{rl}{\Psi }_{\square }& =x_{\square }\stackrel{x_{\square }\to 0}{⟶}0\\ {\Psi }_{a\square }& =\frac{x_a(1-x_a^2-x_b^2)x_{\square }}{1-2x_a^2-2x_b^2+(x_a^2-x_b^2{)}^2}\stackrel{x_{\square }\to 0}{⟶}\frac{x_a}{4}\\ {\Psi }_{aba\square }& =\frac{x_a^2{x}_b{x}_{\square }}{1-2x_a^2-2x_b^2+(x_a^2-x_b^2{)}^2}\stackrel{x_{\square }\to 0}{⟶}\frac{x_a}{8}\\ {\Psi }_{abab\square }& =\frac{x_a^2{x}_b^2{x}_{\square }}{1-2x_a^2-2x_b^2+(x_a^2-x_b^2{)}^2}\stackrel{x_{\square }\to 0}{⟶}\frac{x_a{x}_b}{8}\\ {\Psi }_{a^2\square }& =\frac{x_a^2(1-x_a^2)x_{\square }}{1-2x_a^2-2x_b^2+(x_a^2-x_b^2{)}^2}\stackrel{x_{\square }\to 0}{⟶}\frac{x_a(1+x_a)}{8}\\ {\Psi }_{a^{2}b\square }& =\frac{x_a^2{x}_b{x}_{\square }}{1-2x_a^2-2x_b^2+(x_a^2-x_b^2{)}^2}\stackrel{x_{\square }\to 0}{⟶}\frac{x_a}{8}\\ {\Psi }_{a^{2}ba\square }& =\frac{x_a^3{x}_b{x}_{\square }}{1-2x_a^2-2x_b^2+(x_a^2-x_b^2{)}^2}\stackrel{x_{\square }\to 0}{⟶}\frac{x_a^2}{8}.\end{array}$$

$${\Psi }_w^{\mathcal{M}(\mathsf{KR}(S,A))}=\sum _{\begin{array}{c}s\in \mathcal{S}(S,A)\\ [s{]}_{\mathsf{KR}(S,A)}=w\end{array}}\prod _{a\in s}{x}_a\text{for all w∈K(KR(S,A)).}$$

$${\Psi }_w^{\mathcal{M}(\mathsf{KR}(S,A))}=\sum _{\begin{array}{c}t\in \mathsf{T}\\ [t{]}_{\mathsf{KR}(S,A)}=w\end{array}}\left(\sum _{\begin{array}{c}s\in \mathcal{S}(S,A)\\ \tau (s)=t\end{array}}\prod _{a\in s}{x}_a\right)\text{for all w∈K(KR(S,A)).}$$

$${\Psi }_{\mathsf{Pict}(\mathsf{Mc}\circ \mathsf{KR}(S,A),t)}=\sum _{\begin{array}{c}s\in \mathcal{S}(S,A)\\ \tau (s)=t\end{array}}\prod _{a\in s}{x}_a.$$

Full text

1. Introduction

In our previous paper [RS17], we developed a general theory to compute the stationary distribution of a finite Markov chain. Every finite state Markov chain $\mathcal{M}$ has a random letter representation, that is, a representation of a semigroup $S$ acting on the left on the state space $\Omega$ [LPW09]. Combining the Karnofsky–Rhodes and the McCammond expansion of the right Cayley graph of $S$, we were able to provide a construction of the stationary distribution using finite semigroup theory without the use of linear algebra. The construction relies on the concept of lumping; the distributions for the expanded graphs can be computed thanks to normal forms of the elements. The stationary distribution of the original Markov chain $\mathcal{M}$ is then obtained by lumping.

In this paper, we show that the stationary distribution of any finite Markov chain can be obtained from certain normal (or Gaußian) distributions. The normal distributions are derived from planar graphs by adding directed loops (or circles) to the straight line, which only touch the graph at one point. Let us outline the construction of these normal forms in the remainder of the introduction.

1.1. Straight line

We start with a straight line starting at $\mathbbm{1}$ with $n$ further vertices:

\mathbbm{1}$$1$$2$$\cdots$$n-1$$n

1.2. Adding loops

A loop is a sequence of vertices connected by edges $v_{0}\longrightarrow v_{1}\longrightarrow\cdots\longrightarrow v_{k}$ such that $v_{0}=v_{k}$, but all other vertices $v_{i}$ with $0\leqslant i<k$ are distinct.

Add a loop $\ell$ to any vertex of the straight line constructed in Section 1.1 (except $\mathbbm{1}$) with $k\geqslant 0$ new vertices, which only touches one existing vertex $v$.

\mathbbm{1}$$1$$2$$v$$\cdots$$n-1$$n$$v_{1}$$v_{2}$$\cdots$$v_{k}$$v_{k-1}

The cut of $\ell$ is

v$$v_{1}$$v_{2}$$\cdots$$v_{k-1}$$v_{k}

Continue to add loops at any vertex (except $\mathbbm{1}$), including the new vertices. Multiple loops at a given vertex are allowed.

\mathbbm{1}$$1$$2$$v$$\cdots$$n-1$$n$$v_{1}$$v_{2}$$\vdots$$q$$\cdots$$v_{k}$$v_{k-1}$$q_{1}$$q_{2}$$\vdots$$q_{h}

Let $\overline{G}$ be the directed graph obtained by this procedure. Notice that each such $\overline{G}$ can be drawn in the plane.

1.3. Kleene expressions

Given a finite alphabet $A$, assign a letter $a\in A$ to each arrow in the graph $\overline{G}$. The result is called a loop graph, denoted $G$.

Example 1.1**.**

For the alphabet $A=\{a,b,c,d,x\}$, we might obtain

$G=$    \mathbbm{1}$$1$$2$$3$$4$$1^{\prime}$$2^{\prime}$$a$$b$$c$$x$$b$$a$$d$$c$$a

In general, this procedure gives a non-deterministic automata since different edges emitting from a vertex can be labeled by the same letter. In the above example, vertex 1 has two arrows labeled $b$ coming out of it.

Denote the set of all paths in a loop graph $G$ starting at $\mathbbm{1}$ and ending at $n$ (the last vertex on the initial straight line underlying $G$) by $\mathcal{P}_{G}$. Here a path is given by

[TABLE]

where $v_{i}$ are vertices in $G$ and $a_{i}\in A$ are the labels on the edges.

There is a simple inductive way to describe $\mathcal{P}_{G}$ using Kleene expressions. Given a set $L$, define $L^{0}=\{\varepsilon\}$ given by the empty string, $L^{1}=L$, and recursively $L^{i+1}=\{wa\mid w\in L^{i},a\in L\}$ for each integer $i>0$. Then the Kleene star is

[TABLE]

A Kleene expression only involves letters in $A$, unions, and $\star$. To obtain a Kleene expression for $\mathcal{P}_{G}$, perform the following doubly recursive procedure:

Algorithm 1.

Induction basis: Start at vertex $\mathbbm{1}$ and with the empty expression $L$.

Induction step: Suppose one is at vertex $i\neq n$ (or $\mathbbm{1}$) on the straight line path underlying $G$.

1. (1)

Continue to the next vertex $i+1$ (or $1$) on the straight line path underlying $G$ and append the label $a$ on the edge from $i\stackrel{{\scriptstyle a}}{{\longrightarrow}}i+1$ (or $\mathbbm{1}\stackrel{{\scriptstyle a}}{{\longrightarrow}}1$) to $L$. 2. (2)

If there are loops $\ell_{1},\ell_{2},\ldots,\ell_{k}$ at vertex $i+1$ (or $1$), append the formal expression

[TABLE]

to $L$. The loops $\ell_{1},\ell_{2},\ldots,\ell_{k}$ are in one-to-one correspondence with the edges coming into vertex $i+1$. 3. (3)

If $i+1\neq n$, continue with the next induction step. Else stop and output $L$.

Algorithm 2. For each symbol $\ell_{i}$ in the expression for $L$, do the following:

1. (1)

Consider the loop $\ell_{i}=\left(v_{0}\stackrel{{\scriptstyle a_{1}}}{{\longrightarrow}}v_{1}\stackrel{{\scriptstyle a_{2}}}{{\longrightarrow}}\cdots\stackrel{{\scriptstyle a_{k}}}{{\longrightarrow}}v_{k}=v_{0}\right)$ from vertex $v_{0}$ to $v_{0}$ in $G$. Consider the subgraph of $G$ with straight line $v_{1}\stackrel{{\scriptstyle a_{2}}}{{\longrightarrow}}\cdots\stackrel{{\scriptstyle a_{k}}}{{\longrightarrow}}v_{k}$ and all further loops that are attached to any of the vertices $v_{i}$ in $G$. Attach $\mathbbm{1}$ to $v_{1}$. The resulting graph $G^{(i)}$ is a new loop graph. Perform Algorithm 1 on $G^{(i)}$ to obtain a Kleene expression $L^{(i)}$. Replace the symbol $\ell_{i}$ in $L$ by $L^{(i)}$. 2. (2)

Continue this process until $L$ does not contain any further expressions $\ell_{i}$ for some loop $\ell_{i}$, that is, $L$ only contains unions, $\star$ and elements in the alphabet $A$. Then the Kleene expression for $\mathcal{P}_{G}$ is $L$.

The resulting expressions can be made into unionless expressions by using Zimin words

[TABLE]

Expressions for larger unions can be obtained by induction using (1.1).

Example 1.2**.**

Let $G$ be as in Example 1.1. Then

[TABLE]

where $\ell_{1}$ is the loop attached to vertex 1. Cut this loop and continue the process to obtain

[TABLE]

where $\ell_{1}^{\prime}$ is the loop at vertex $1^{\prime}$ labelled $a$ and $\ell_{2}^{\prime}$ is the loop at vertex $1^{\prime}$ labelled $c$. We have $\ell_{1}^{\prime}=a$ and $\ell_{2}^{\prime}=c$, so that altogether we find

[TABLE]

where in the last step we used the Zimin words to get rid of the unions. This is a Kleene expression for $\mathcal{P}_{G}$.

See Example 3.8 for another example and also compare this construction to the definition of $\mathsf{Pict}$ in Definition 3.5.

Main results

We are now going to define normal distributions.

Definition 1.3** (Normal distribution).**

Let $G$ be a loop graph with edges labeled by letters in the alphabet $A$. Associate the indeterminate $x_{a}$ to $a\in A$. Then the normal distribution of $G$ is defined as

[TABLE]

We may use the Kleene expressions of the previous section for $\mathcal{P}_{G}$. The advantage in doing so is that one can immediately obtain rational expressions. Namely, using the geometric series, we find that

[TABLE]

Similarly

[TABLE]

In general, using the recursion (1.1) we derive by induction

[TABLE]

Our main theorem is the following.

Theorem 1.4**.**

The stationary distribution $\Psi^{\mathcal{M}}$ of a finite Markov chain $\mathcal{M}$ is the sum of normal distributions $\Psi_{G}$ or certain limits of $\Psi_{G}$, where $G$ is a loop graph.

The proof of Theorem 1.4 is given in Section 3.3. A more precise version of Theorem 1.4 is stated in Theorem 3.9.

The paper is outlined as follows. In Section 2, we review the main results from [RS17], in particular the expressions for the stationary distribution of a finite Markov chain in terms of semaphore codes of the Karnofsky–Rhodes expansion of the right Cayley graph of the underlying semigroup. In Section 3, we review the McCammond expansion and its relation to semaphore codes and provide the definition of $\mathsf{Pict}$. The map $\mathsf{Pict}$ is used to give a proof of Theorem 1.4. The original definition of $\mathsf{Pict}$ is due to McCammond, but the applications to random walks are due to the authors.

Acknowledgments

We are grateful to Jon McCammond and Ben Steinberg for discussions. The map $\mathsf{Pict}$ of Definition 3.5 is due to McCammond, told to the first author in 1994, written by the first author in 2008, and simplified here.

The first author thanks the Simons Foundation Collaboration Grants for Mathematicians for travel grant #313548. The second author was partially supported by NSF grants DMS–1760329 and DMS–1764153.

2. Stationary distributions of Markov chains

In this section, we provide definitions and review the necessary results we need from [RS17].

2.1. Markov chains

A Markov chain $\mathcal{M}$ consists of a finite or countable state space $\Omega$ together with transition probabilities $\mathcal{T}_{s^{\prime},s}$ for the transition $s\longrightarrow s^{\prime}$ for $s,s^{\prime}\in\Omega$. The matrix $\mathcal{T}=(\mathcal{T}_{s^{\prime},s})_{s,s^{\prime}\in\Omega}$ is called the transition matrix, which is a column-stochastic matrix, meaning that the column sums of $\mathcal{T}$ are equal to one.

A Markov chain is irreducible if for any $s,s^{\prime}\in\Omega$ there exists an integer $m$ (possibly depending on $s$, $s^{\prime}$) such that $\mathcal{T}_{s^{\prime},s}^{m}>0$. In other words, one can get from any state $s$ to any other state $s^{\prime}$ using only steps with positive probability. A state $s\in\Omega$ is called recurrent if the system returns to $s$ in finitely many steps with probability one.

The stationary distribution of $\mathcal{M}$ is a vector $\Psi=(\Psi_{s})_{s\in\Omega}$ such that $\mathcal{T}\Psi=\Psi$ and $\sum_{s\in\Omega}\Psi_{s}=1$. In other words, $\Psi$ is a right-eigenvector of $\mathcal{T}$ with eigenvalue one. If the Markov chain is irreducible, the stationary distribution is unique [LPW09].

Next we define lumping of Markov chains. Partition the state space $\Omega$ into $(\Omega_{1},\ldots,\Omega_{\ell})$ such that

[TABLE]

One may view such a partition as an equivalence relation $s\sim s^{\prime}$ if $s,s^{\prime}\in\Omega_{i}$ for some $1\leqslant i\leqslant\ell$. We say that $\mathcal{M}$ can be lumped with respect to the partition $(\Omega_{1},\ldots,\Omega_{\ell})$ if the transition matrix $\mathcal{T}$ satisfies [LPW09, Lemma 2.5] [KS76] for all $1\leqslant i,j\leqslant\ell$

[TABLE]

The lumped Markov chain is a random walk on the equivalence classes, whose stationary distribution labeled by $w$ is $\sum_{s\sim w}\Psi_{s}$.

Every finite state Markov chain $\mathcal{M}$ has a random letter representation, that is, a representation of a semigroup $S$ acting on the left on the state space $\Omega$ (see [LPW09, Proposition 1.5] and [ASST15, Theorem 2.3]). In this setting, we transition $s\stackrel{{\scriptstyle a}}{{\longrightarrow}}s^{\prime}$ with probability $0\leqslant x_{a}\leqslant 1$, where $s,s^{\prime}\in\Omega$, $a\in S$ and $s^{\prime}=a.s$ is the action of $a$ on the state $s$. Let $A=\{a\in S\mid x_{a}>0\}$. We assume that $A$ generates $S$; if not, it suffices to consider the subsemigroup generated by $A$. Note that $\sum_{a\in A}x_{a}=1$. The transition matrix $\mathcal{T}$ of $\mathcal{M}$ is the $|\Omega|\times|\Omega|$-matrix

[TABLE]

Note that we may assume that the action of $S$ on $\Omega$ is faithful as this does not affect the random walk.

If $S$ is a semigroup, then $S^{\mathbbm{1}}$ denotes $S$ with an adjoint identity $\mathbbm{1}$ even if $S$ already has an identity.

Definition 2.1** (Ideal).**

Let $S$ be a semigroup. A two-sided ideal $I$ (or ideal for short) is a subset $I\subseteq S$ such that $uIv\subseteq I$ for all $u,v\in S^{\mathbbm{1}}$. Similarly, a left ideal $I$ is a subset $I\subseteq S^{\mathbbm{1}}$ such that $uI\subseteq I$ for all $u\in S^{\mathbbm{1}}$.

If $I,J$ are ideals of $S$, then $IJ\subseteq I\cap J$, so that $I\cap J\neq\emptyset$. Hence every finite semigroup has a unique minimal ideal denoted $K(S)$. As shown in [CP61, KRT68], the minimal ideal $K(S)$ of a finite semigroup $S$ is the disjoint union of all the minimal left ideals of $S$ and the Rees Theorem applies. By [ASST15, Remark 2.8] the faithful left action of $S$ on $\Omega$ is isomorphic to the left action of $S$ on $K(S)$.

Let $(S,A)$ be a semigroup $S$ together with a choice of generators $A$ for $S$. Define $\mathcal{M}(S,A)$ to be the Markov chain, where the transition $s\stackrel{{\scriptstyle a}}{{\longrightarrow}}s^{\prime}$ for $s,s^{\prime}\in S$ and $a\in A$ is given by $s^{\prime}=as$ in the left Cayley graph with probability $0<x_{a}\leqslant 1$. Note that we are assuming that all probabilities $x_{a}$ for $a\in A$ are nonzero. Then it was shown in [HM11] (see also [ASST15, Proposition 3.2]) that the recurrent states of $\mathcal{M}(S,A)$ are the elements in $K(S)$. Furthermore, the connected components of the recurrent states in the random walk are the minimal left ideals of $S$. The restriction of the random walk to any minimal left ideal is irreducible. Moreover, the chain so obtained is independent of the chosen minimal left ideal. This random walk and the random walk with states a left ideal $L$ of $K(S)$ and $S$ acting on the left made faithful, that is $x\stackrel{{\scriptstyle a}}{{\longrightarrow}}y$ for $x\in L$ and $y=ax$, are essentially the same. So we may not distinguish the two cases.

2.2. Karnofsky–Rhodes expansion

In this section, we define the right Cayley graph of a finite semigroup and its Karnofsky–Rhodes expansions.

Definition 2.2** (Right Cayley graph).**

Let $(S,A)$ be a finite semigroup $S$ together with a set of generators $A$. The right Cayley graph $\mathsf{RCay}(S,A)$ of $S$ with respect to $A$ is the rooted graph with vertex set $S^{\mathbbm{1}}$, root $r=\mathbbm{1}\in S^{\mathbbm{1}}$, and edges $s\stackrel{{\scriptstyle a}}{{\longrightarrow}}s^{\prime}$ for all $(s,a,s^{\prime})\in S^{\mathbbm{1}}\times A\times S^{\mathbbm{1}}$, where $s^{\prime}=sa$ in $S^{\mathbbm{1}}$.

A path $p$ in $\mathsf{RCay}(S,A)$ is a sequence

[TABLE]

where $v_{i}\in S^{\mathbbm{1}}$ are vertices in $\mathsf{RCay}(S,A)$ and $v_{i}\stackrel{{\scriptstyle a_{i}}}{{\longrightarrow}}v_{i+1}$ are edges in $\mathsf{RCay}(S,A)$. The endpoint of $p$ is $\tau(p):=v_{\ell+1}$. The length of the path $p$ is $\ell(p):=\ell$, which equals the number of edges. A simple path is a path that does not visit any vertex twice. Empty paths are considered simple. A path which starts and ends at the same vertex is called a circuit. A circuit that is simple, when the last vertex is removed, is called a loop.

Definition 2.3** (Transition edges).**

An edge $s\stackrel{{\scriptstyle a}}{{\longrightarrow}}s^{\prime}$ in the right Cayley graph $\mathsf{RCay}(S,A)$ is a transition edge if there is no directed path from $s^{\prime}$ to $s$ in $\mathsf{RCay}(S,A)$. In other words, there does not exist any sequence $a_{1},\ldots,a_{k}\in A$ with $k\geqslant 1$ such that $s^{\prime}(a_{1}\cdots a_{k})=s$.

Let us now define the Karnofsky–Rhodes expansion of the right Cayley graph (see also [MRS11, Definition 4.15] and [MSS15, Section 3.4]). Let $(A^{+},A)$ be the free semigroup with generators $A$, where $A^{+}$ is the set of all words $a_{1}\ldots a_{\ell}$ of length $\ell\geqslant 1$ over $A$ with multiplication given by concatenation. When we write $[a_{1}\cdots a_{\ell}]_{S}$, we mean the element in $S$ when taking the product in the semigroup of the generators $a_{i}\in A$.

Definition 2.4** (Karnofksy–Rhodes expansion).**

The Karnofsky–Rhodes expansion $\mathsf{KR}(S,A)$ is obtained as follows. Start with the right Cayley graph $\mathsf{RCay}(A^{+},A)$. Identify two paths in $\mathsf{RCay}(A^{+},A)$

[TABLE]

in $\mathsf{KR}(S,A)$ if and only if the corresponding paths in $\mathsf{RCay}(S,A)$

[TABLE]

where $v_{i}=a_{1}a_{2}\ldots a_{i}$ and $v_{i}^{\prime}=a_{1}^{\prime}a_{2}^{\prime}\ldots a^{\prime}_{i}$, end at the same vertex $[v_{\ell}]_{S}=[v^{\prime}_{\ell^{\prime}}]_{S}$ and in addition the set of transition edges of $[p]_{S}$ and $[p^{\prime}]_{S}$ in $\mathsf{RCay}(S,A)$ is equal.

Example 2.5**.**

Consider the right Cayley graph of the Klein $4$-group $Z_{2}\times Z_{2}$ with zero with generators $\{a,b,\square\}$, where $a=(1,-1)$, $b=(-1,1)$, and $\square$ is the zero. The right Cayley graph $\mathsf{RCay}(Z_{2}\times Z_{2}\cup\{\square\},\{a,b,\square\})$ is

\mathbbm{1}$$(1,-1)$$(-1,1)$$(-1,-1)$$(1,1)$$\square$$a$$b$$b$$a$$a$$b$$\square$$\square$$\square$$\square$$\square

where all three arrows $a,b,\square$ fix the vertex $\square$ at the bottom. Transition edges are indicated in blue. Double edges mean that right multiplication by the label for either vertex yields the other vertex. The Karnofsky–Rhodes expansion of this right Cayley graph is given by

\mathbbm{1}$$a$$b$$ab$$ba$$a^{2}$$b^{2}$$a^{2}b=aba$$bab=b^{2}a$$\square$$a\square$$ab\square$$a^{2}b\square$$a^{2}\square$$b\square$$ba\square$$b^{2}a\square$$b^{2}\square$$a b$$a b$$b a$$a $b$ b$$a$$\square$$\square$$\square$$\square$$\square$$\square$$\square$$\square$$\square$$\square

where arrows $a,b,\square$ fix all the vertices at the bottom.

Proposition 2.6**.**

[RS17*, Proposition 2.15]**

$\mathsf{KR}(S,A)$ is the right Cayley graph of a semigroup, also denoted by $\mathsf{KR}(S,A)$.*

2.3. Stationary distribution

We now review the main results of [RS17], which give the stationary distribution for any Markov chain $\mathcal{M}(S,A)$ for a finite semigroup with chosen generators $(S,A)$. Recall that $\mathcal{M}(S,A)$ is the random walk on the unique minimal ideal $K(S)$ of $S$. More precisely, the random walk is given by the left action of $S$ on $K(S)$.

To state our results for the stationary distribution, we first need to review the semaphore codes associated to $(S,A)$ [BPR10]. The semaphore code $\mathcal{S}(S,A)$ is the set of all words $a_{1}a_{2}\ldots a_{\ell}\in A^{+}$ such that $[a_{1}a_{2}\cdots a_{\ell}]_{S}\in K(S)$, but $[a_{1}a_{2}\cdots a_{\ell-1}]_{S}\not\in K(S)$.

The main results are the following.

Theorem 2.7**.**

[RS17*, Corollary 2.28]**

The Markov chain $\mathcal{M}(S,A)$ is the lumping of $\mathcal{M}(\mathsf{KR}(S,A))$ with stationary distribution*

[TABLE]

The next result is non-trivial. It requires the assumption that the minimal ideal $K(S)$ is left zero, that is, $xy=x$ for all $x,y\in K(S)$.

Theorem 2.8**.**

[RS17*, Theorem 2.12]**

If $K(S)$ is left zero, the stationary distribution of the Markov chain $\mathcal{M}(\mathsf{KR}(S,A))$ is given by*

[TABLE]

As outlined in [RS17, Section 2.9], the case when $K(S)$ is not left zero can be constructed from the case when $K(S)$ is left zero using the flat operation. That is, one adds an additional generator $\square$ to the alphabet $A$, which acts as zero. The associated probability is $x_{\square}$. The elements in the minimal ideal $K(\mathsf{KR}(S\cup\{\square\},A\cup\{\square\}))$ are of the form $w\square$, where $w\in\mathsf{KR}(S,A)$. Since $\square v=\square$ for all $v\in\mathsf{KR}(S,A)$, we indeed have that $K(\mathsf{KR}(S\cup\{\square\},A\cup\{\square\}))$ is left zero and hence Theorem 2.8 applies. Then [RS17, Corollary 2.33]

[TABLE]

3. Normal distributions for random walks

In this section, we prove Theorem 1.4. By Theorems 2.7 and 2.8 and Equation (2.3), the stationary distribution $\Psi_{w}^{\mathcal{M}(S,A)}$ is the sum of terms of the form $\prod_{a\in s}x_{a}$, where $s\in\mathcal{S}(S,A)$ (or limits of such expressions). In Section 3.1, we will explain how the semaphore code $\mathcal{S}(S,A)$ is related to the McCammond expansion $\mathsf{Mc}\circ\mathsf{KR}(S,A)$. In Section 3.2, we will then define the map $\mathsf{Pict}$ on $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ to deduce that $\Psi_{w}^{\mathcal{M}(S,A)}$ is a sum of normal forms. A proof of Theorem 1.4 is given in Section 3.3. Theorem 3.9 is a more precise version of Theorem 1.4.

3.1. The McCammond expansion and semaphore codes

Let us now turn to the McCammond expansion [McC01, MRS11] of the Karnofsky–Rhodes expansion of the right Cayley graph of $(S,A)$. Recall that a simple path in $\mathsf{KR}(S,A)$ is a path that does not visit any vertex twice. Empty paths are considered simple.

Definition 3.1** (McCammond expansion).**

The McCammond expansion $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ of $\mathsf{KR}(S,A)$ is the graph with vertex set $V$, which is the set of simple paths in $\mathsf{KR}(S,A)$. The edges are given by

[TABLE]

In other words, if the path $pa$ in $\mathsf{KR}(S,A)$ is simple, then $q=pa$. Otherwise $\tau(pa)=v$ is a vertex of $p$ and then $q$ is the initial segment of $p$ up to and including $v$.

Remark 3.2**.**

Note that $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ has a spanning tree $\mathsf{T}$ with the same vertex set as $\mathsf{Mc}\circ\mathsf{KR}(S,A)$, but only those edges $(p,a,q)\in E$ such that $\ell(q)=\ell(p)+1$.

Example 3.3**.**

The McCammond expansion of $\mathsf{KR}(S,A)$ of Example 2.5 is given in Figure 1.

By Remark 3.2, the McCammond expansion $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ has a spanning tree $\mathsf{T}$. In this tree, the vertices are naturally labeled by the sequence of edge labels in the path from $\mathbbm{1}$ to the vertex. More concretely, if

[TABLE]

is a path in $\mathsf{T}$, then the vertex $v_{\ell}$ is naturally labeled by $a_{1}\ldots a_{\ell}$. Hence the corresponding vertex $v_{\ell}$ in $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ has a normal form given by $a_{1}\ldots a_{\ell}$.

Remark 3.2 also ensures that $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ has the unique simple path property, defined as follows.

Definition 3.4** (Unique simple path property).**

A rooted graph $(\Gamma,\mathbbm{1})$ with root $\mathbbm{1}$ has the unique simple path property if for each vertex $v$ in $\Gamma$ there is a unique simple path from the root $\mathbbm{1}$ to $v$.

Elements in the semaphore code $\mathcal{S}(S,A)$ are paths in $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ (rather than in $\mathsf{T}$) starting at $\mathbbm{1}$ and ending in $K(S)$. They are also in natural correspondence with words $a_{1}\ldots a_{\ell}\in A^{+}$ such that $[a_{1}\cdots a_{\ell}]_{S}\in K(S)$ and $[a_{1}\cdots a_{\ell-1}]_{S}\not\in K(S)$. From the semaphore code, one can obtain the normal form by stripping away all loops in the path.

3.2. Definition of $\mathsf{Pict}$

We are now going to define the map $\mathsf{Pict}$ from the set of tuples $(\Gamma,p)$, where $\Gamma$ is a graph with the unique simple path property and $p$ is a simple path in $\Gamma$ starting at $\mathbbm{1}$, to the set of loop graphs. The straight line, that the loop graph is based on, will correspond to $p$. The map $\mathsf{Pict}$ was first defined by McCammond (we give a simplified definition here).

Definition 3.5** (McCammond).**

Let $\Gamma$ be a graph with the unique simple path property and $p$ a simple path in $\Gamma$ starting at $\mathbbm{1}$. Then $\mathsf{Pict}(\Gamma,p)$ is defined by the principle of induction.

Induction basis: Set $P=p$ and start at vertex $v_{0}=\mathbbm{1}$.

Induction step: Suppose one is at vertex $v_{0}\neq\tau(p)$ on path $p$. Take the edge $e$ from $v_{0}$ to $v_{1}$ in $p$.

1. (1)

If there is no edge in $\Gamma$ coming into $v_{1}$ besides $e$, continue with the unique next vertex in $p$, now denoted $v_{1}$ (with the current vertex $v_{1}$ relabeled $v_{0}$), unless $v_{1}=\tau(p)$. If $v_{1}=\tau(p)$, then output $\mathsf{Pict}(\Gamma,p)=P$. 2. (2)

Otherwise there is at least one edge $e^{\prime}\neq e$ in $\Gamma$ going into $v_{1}$, given by $e^{\prime}=\left(v^{\prime}\stackrel{{\scriptstyle a}}{{\longrightarrow}}v_{1}\right)$ for some $a\in A$. Since $\Gamma$ has the unique simple path property by assumption, there must be a unique simple path starting at $\mathbbm{1}$ going to $v_{0}$ along the path $p$ followed by the path $p^{\prime}$ starting at $v_{0}$, going along $e$ to $v_{1}$, and ending at $v^{\prime}$.

1. (a)

Run the induction on $p^{\prime}$ in a subgraph $\Gamma^{\prime}$ of $\Gamma$, consisting of all edges and vertices on circuits containing a vertex of $p^{\prime}$. Note that $p^{\prime}$ is simple in $\Gamma^{\prime}$. The output is $P^{\prime}=\mathsf{Pict}(\Gamma^{\prime},p^{\prime})$. 2. (b)

Modify $P$ by attaching $P^{\prime}$ disjointly except at $v_{1}$ and adding edge $e^{\prime}$ from $v^{\prime}$ in $P^{\prime}$ back to $v_{1}$. 3. (3)

Repeat step (2) for each edge $e^{\prime}\neq e$ at vertex $v_{1}$. 4. (4)

Continue with the induction step unless $v_{1}=\tau(p)$. If $v_{1}=\tau(p)$, then output $\mathsf{Pict}(\Gamma,p)=P$.

Remark 3.6**.**

If $\Gamma$ is a rooted graph with the unique simple path property, then $\Gamma$ with some edges removed (and any vertices that are no longer connected to the root $\mathbbm{1}$) still has the unique simple path property. This is the case since either the unique simple path from $\mathbbm{1}$ to $v$ is still there or the vertex $v$ is now disconnected from $\mathbbm{1}$ and has hence been removed.

The graph $\Gamma^{\prime}$ in the Induction step (2)(a) in the definition of $\mathsf{Pict}$ can be obtained in two steps. First remove all incoming and outgoing edges on the vertices along the path $p$ from $\mathbbm{1}$ to $v_{1}$, except the edges on the path $p$ itself. Remove all vertices that have become disconnected in this process. By the remark above, the resulting graph still has the unique simple path property. In this graph, all simple paths go through the vertex $v_{1}$. Hence we may make $v_{0}$ the root (removing all vertices $\mathbbm{1}$ up to $v_{0}$ along $p$). The result is $\Gamma^{\prime}$, which still has the unique simple path property.

Example 3.7**.**

Let $p=\left(\mathbbm{1}\stackrel{{\scriptstyle a}}{{\longrightarrow}}1\stackrel{{\scriptstyle b}}{{\longrightarrow}}2\stackrel{{\scriptstyle c}}{{\longrightarrow}}3\right)$ in

$\Gamma=$    \mathbbm{1}$$1$$2$$3$$4$$a$$b$$c$$a$$d$$a

To compute $\mathsf{Pict}(\Gamma,p)$, we start with $P=p$, $v_{0}=\mathbbm{1}$ and $v_{1}=1$. We are in step (2) of the Induction step with $e=\left(\mathbbm{1}\stackrel{{\scriptstyle a}}{{\longrightarrow}}1\right)$ and $e^{\prime}=\left(4\stackrel{{\scriptstyle a}}{{\longrightarrow}}1\right)$. Then $p^{\prime}=\left(\mathbbm{1}\stackrel{{\scriptstyle a}}{{\longrightarrow}}1\stackrel{{\scriptstyle b}}{{\longrightarrow}}2\stackrel{{\scriptstyle d}}{{\longrightarrow}}4\right)$ and $\Gamma^{\prime}$ is $\Gamma$ with the arrow labelled $a$ from $v^{\prime}=4$ to $v_{1}=1$ removed. Also $P^{\prime}=\mathsf{Pict}(\Gamma^{\prime},p^{\prime})$ is $p^{\prime}$ with a loop labelled $a$ at vertex $2$. Attaching $P^{\prime}$ at $v_{1}=1$ (with its vertex $2$ relabelled to $2^{\prime}$ to avoid repetition) and adding edge $e^{\prime}$ we obtain

$P=$    \mathbbm{1}$$1$$2$$3$$4$$2^{\prime}$$a$$b$$c$$a$$b$$d$$a

Since there are no further edges going into vertex $v_{1}=1$, we continue with the induction along $p$. This means that we set $v_{0}=1$, $v_{1}=2$, and $e=\left(1\stackrel{{\scriptstyle b}}{{\longrightarrow}}2\right)$. Besides $e$, there is only one other arrow going into $v_{1}=2$ in $\Gamma$, namely $e^{\prime}=\left(2\stackrel{{\scriptstyle a}}{{\longrightarrow}}2\right)$. In this case $p^{\prime}=1\stackrel{{\scriptstyle b}}{{\longrightarrow}}2$ and $\Gamma^{\prime}$ is $\Gamma$ with $\mathbbm{1}$ and the arrows $\mathbbm{1}\stackrel{{\scriptstyle a}}{{\longrightarrow}}1$, $4\stackrel{{\scriptstyle a}}{{\longrightarrow}}1$, and $2\stackrel{{\scriptstyle a}}{{\longrightarrow}}2$ removed. Hence the new $P$ with $P^{\prime}=\mathsf{Pict}(\Gamma^{\prime},p^{\prime})$ added is

$\mathsf{Pict}(\Gamma,p)=P=$    \mathbbm{1}$$1$$2$$3$$4$$2^{\prime}$$a$$b$$c$$a$$b$$d$$a$$a

The remaining induction steps do not change this $P$, which is hence also $\mathsf{Pict}(\Gamma,p)$.

Example 3.8**.**

Consider the McCammond expansion $\Gamma=\mathsf{Mc}\circ\mathsf{KR}(S,A)$ of Example 3.3 (see also Figure 1) and the path in the McCammond tree $\mathsf{T}$ given by $ab\square$. Then $\mathsf{Pict}(\Gamma,ab\square)$ is given by

\mathbbm{1}$$a$$ab$$ab\square$$a$$b$$\square$$\bullet$$\bullet$$a$$a$$b$$b$$\bullet$$\bullet$$\bullet$$a$$a$$b$$b$$a$$a$$\bullet$$\bullet$$\bullet$$b$$b$$a$$a$$b$$b$$\bullet$$\bullet$$\bullet$$a$$b$$a$$b$$\bullet$$\bullet$$\bullet$$b$$a$$b$$a$$\bullet$$a$$a$$\bullet$$b$$b$$\bullet$$b$$b$$\bullet$$a$$a$$\bullet$$a$$a$$\bullet$$b$$b

Following the algorithm explained in Section 1.3, a Kleene expression for $\mathcal{P}_{\mathsf{Pict}(\Gamma,ab\square)}$ is given by

[TABLE]

where

[TABLE]

Hence

[TABLE]

Using that $x_{a}+x_{b}+x_{\square}=1$, we find that in the limit $x_{\square}\to 0$

[TABLE]

In a similar fashion, we find

[TABLE]

The stationary probabilities for the elements with $a$ and $b$ interchanged are obtained by symmetry. It is not hard to check that these probabilities sum to one as desired.

As noted in the introduction, $\mathsf{Pict}(\Gamma,p)$ is not necessarily deterministic. There can be several arrows leaving a vertex labeled by the same element $a\in A$. For example, vertex $1$ in Example 3.7 has two arrows labeled $b$ coming out.

One can make a non-deterministic automata $\mathcal{A}$ deterministic as follows. If $\mathcal{A}$ has states $Q$ with start state $\mathbbm{1}$ and final states $F$ not containing $\mathbbm{1}$, we make a deterministic automata $\mathsf{det}(\mathcal{A})$ accepting the same strings going from $\mathbbm{1}$ to a member of $F$ as follows. The states $Q^{\prime}$ of $\mathsf{det}(\mathcal{A})$ are the collection of subsets of $Q$ determined a follows:

• •

$\{\mathbbm{1}\}$ is in $Q^{\prime}$;

• •

if $Z\in Q^{\prime}$, then $Z.a\in Q^{\prime}$ for $a\in A$, where $Z.a=\{q\mid z\stackrel{{\scriptstyle a}}{{\longrightarrow}}q\in\mathcal{A}\text{ where }z\in Z\}$.

One continues by induction until the process adds no new subsets. For $\mathsf{det}(\mathcal{A})$, start in state $\{\mathbbm{1}\}$. The final states are all the states of $\mathsf{det}(\mathcal{A})$ such that the intersection with $F$ is non-empty.

With this definition, making $\mathsf{Pict}(\Gamma,p)$ deterministic gives the automata for $(\Gamma,p)$ back.

3.3. Proof of Theorem 1.4

As explained in Section 2.1, any finite Markov chain $\mathcal{M}$ can be described as a Markov chain $\mathcal{M}(S,A)$ in terms of a finite semigroup $S$ with generators $A$. Since by Theorem 2.7, $\Psi_{w}^{{\mathcal{M}(S,A)}}$ is the sum over $\Psi_{v}^{{\mathcal{M}(\mathsf{KR}(S,A))}}$, it suffices to prove the statement of Theorem 1.4 for $\Psi_{v}^{{\mathcal{M}(\mathsf{KR}(S,A))}}$. When $K(S)$ is not left zero, we may use the limiting construction of (2.3) to obtain $\Psi_{v}^{{\mathcal{M}(\mathsf{KR}(S,A))}}$ from the case in which the minimal ideal is left zero. Assuming that $K(S)$ is left zero, we have by Theorem 2.8

[TABLE]

As explained in Section 3.1, there is a normal form associated to each semaphore code element $s\in\mathcal{S}(S,A)$. Namely, $s$ is a path in $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ starting at $\mathbbm{1}$ and the normal form is the simple path with all loops stripped away from $s$; equivalently the normal form is the path in $\mathsf{T}$ starting at $\mathbbm{1}$ and ending at $\tau(s)$, where $\mathsf{T}$ is the tree associated to the McCammond expansion $\mathsf{Mc}\circ\mathsf{KR}(S,A)$. In the tree $\mathsf{T}$, a path $p$ starting at $\mathbbm{1}$ is also naturally in bijection with its endpoint $\tau(p)$. Hence we may identify vertex $t\in\mathsf{T}$ with the path from $\mathbbm{1}$ to $t$ in $\mathsf{T}$ or equivalently with the simple path from $\mathbbm{1}$ to $t$ in $\mathsf{Mc}\circ\mathsf{KR}(S,A)$. Therefore, we may rewrite the sum in (3.1) as

[TABLE]

We claim that for a given $t\in\mathsf{T}$ with $[t]_{\mathsf{KR}(S,A)}\in K(\mathsf{KR}(S,A))$

[TABLE]

Recall that by Definition 1.3

[TABLE]

Hence (3.3) can be proved by establishing a bijection

[TABLE]

In fact, we are going to prove a slight generalization of (3.4). Namely, for any $t\in\mathsf{T}$ we will show that there is a bijection

[TABLE]

where $\mathcal{P}_{\mathsf{Mc}\circ\mathsf{KR}(S,A)}$ is the set of paths in $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ starting at $\mathbbm{1}$. Then (3.4) is the special case when $[t]_{\mathsf{KR}(S,A)}\in K(\mathsf{KR}(S,A))$.

To define $\varphi$ in (3.5), fix $t=a_{1}\cdots a_{k}$, where $a_{i}\in A$ are the labels in the path in $\mathsf{T}$. A path $s\in\mathcal{P}_{\mathsf{Mc}\circ\mathsf{KR}(S,A)}$ with $\tau(s)=t$, can be viewed as $t$ with circuits $\ell_{j}^{(j)}$ interspersed. More precisely,

[TABLE]

where $\tau(a_{1}\cdots a_{i})=\tau(a_{1}\cdots a_{i}\ell_{i}^{(j)})$ for all $1\leqslant i\leqslant k$ and $j\in J_{i}$ and any initial subsequence of $\ell_{i}^{(j)}$ does not reach the vertex $a_{1}\cdots a_{i}$. Here the sets $J_{i}$ index the set of circuits $\{\ell_{i}^{(j)}\mid j\in J_{i}\}$ at vertex $a_{1}\cdots a_{i}$ and either $J_{i}=\{1,2,\ldots,n_{i}\}$ is a finite set or $J_{i}=\{1,2,3,\ldots\}$ is the set of positive integers. In other words, each $\ell_{i}^{(j)}$ is a circuit from vertex $a_{1}\cdots a_{i}$ to itself, which does not pass through $a_{1}\cdots a_{i}$ otherwise. The last step of $\ell_{i}^{(j)}$ is an edge in $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ that is not in $\mathsf{T}$. Suppose by induction that

[TABLE]

where $1\leqslant i\leqslant k$ and $J^{\prime}_{i}=\{1,2,\ldots,n_{i}^{\prime}\}\subseteq J_{i}$ or $J_{i}^{\prime}=J_{i}$, is mapped to $\pi$ in $\mathsf{Pict}(\mathsf{Mc}\circ\mathsf{KR}(S,A),a_{1}\cdots a_{i})$ under $\varphi$. We need to distinguish two cases.

Case $J_{i}^{\prime}\subsetneq J_{i}$. Let $j$ be the smallest element in $J_{i}\setminus J_{i}^{\prime}$. Recall that $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ has the unique simple path property. Hence the path $p^{\prime}$ in $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ from $v_{0}=a_{1}\cdots a_{i-1}$ through $v_{1}=a_{1}\cdots a_{i}$ to $v^{\prime}$, which is $a_{1}\cdots a_{i}\ell_{i}^{(j)}$ with the last edge $e^{\prime}$ removed is a path in $\Gamma^{\prime}$ in the notation of Section 3.2. By induction this path is mapped to $\pi^{\prime}$ in $\mathcal{P}_{\mathsf{Pict}(\Gamma^{\prime},p^{\prime})}$. Hence

[TABLE]

This corresponds to the induction step (2) in Definition 3.5.

Case $J_{i}^{\prime}=J_{i}$. If $i=k$, we are done. If $i<k$, we define

[TABLE]

which is a well-defined path since the last step is along the straight line path and hence unique. This corresponds to the induction step (1) (if $J_{i}=\emptyset$) or step (4) (if $J_{i}\neq\emptyset$) in Definition 3.5.

This shows that $\varphi$ is a well-defined map. It has an inverse $\varphi^{-1}$ by mapping a path $\pi\in\mathcal{P}_{\mathsf{Pict}(\mathsf{Mc}\circ\mathsf{KR}(S,A),t)}$ to a path in $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ by just reading the labels of the edges. This indeed gives a path in $\mathsf{Mc}\circ\mathsf{KR}(S,A)$ by the construction of $\mathsf{Pict}$.

Combining (3.2) and (3.3), we obtain

[TABLE]

which proves Theorem 1.4 since $\mathsf{Pict}(\mathsf{Mc}\circ\mathsf{KR}(S,A),t)$ is a loop graph.

In summary, we proved the following theorem, which is a more detailed version of Theorem 1.4.

Theorem 3.9**.**

Let $\mathcal{M}(S,A)$ be a Markov chain associated to the finite semigroup with generators $(S,A)$. If $K(S)$ is left zero, the stationary distribution is given by

[TABLE]

where $\mathsf{T}$ is the spanning tree of $\mathsf{Mc}\circ\mathsf{KR}(S,A)$. Otherwise

[TABLE]

where $\mathsf{T}$ is the spanning tree of $\mathsf{Mc}\circ\mathsf{KR}(S\cup\{\square\},A\cup\{\square\})$ and $\square$ acts as zero.