How strong can the Parrondo effect be?

TL;DR

This paper investigates the potential strength of the Parrondo effect, demonstrating that with arbitrary parameters and sequences, the combined game can yield profits arbitrarily close to 100%.

Contribution

It extends the understanding of the Parrondo effect by showing how parameter choices and game sequences can maximize the profit rate.

Findings

Profit rate can approach 100% with appropriate parameters.

Arbitrary periodic sequences can optimize the Parrondo effect.

The effect's strength is unbounded under certain conditions.

Abstract

If the parameters of the original Parrondo games $A$ and $B$ are allowed to be arbitrary, subject to a fairness constraint, and if the two (fair) games $A$ and $B$ are played in an arbitrary periodic sequence, then the rate of profit can not only be positive, it can be arbitrarily close to 1 (i.e., 100%).

Tables

Table 1. Table 1: The rate of profit μ ​ ( r , 0 , ( A ​ B ) s ​ B r − 2 ) 𝜇 𝑟 0 superscript 𝐴 𝐵 𝑠 superscript 𝐵 𝑟 2 \mu(r,0,(AB)^{s}B^{r-2}) . Here, for a given odd r 𝑟 r , we choose s 𝑠 s to maximize s ′ ↦ μ ​ ( r , 0 , ( A ​ B ) s ′ ​ B r − 2 ) maps-to superscript 𝑠 ′ 𝜇 𝑟 0 superscript 𝐴 𝐵 superscript 𝑠 ′ superscript 𝐵 𝑟 2 s^{\prime}\mapsto\mu(r,0,(AB)^{s^{\prime}}B^{r-2}) . Results are rounded to six significant digits.

$r$	$s$	$\mu (r,0,{\left(AB\right)}^s{B}^{r-2})$		00$r$	0$s$	$\mu (r,0,{\left(AB\right)}^s{B}^{r-2})$
3	2	$9/25=0.360000$		0025	05	0.711662
5	3	$35/81\approx 0.432099$		0125	07	0.898263
7	3	$49/99\approx 0.494949$		0625	09	0.971238
9	3	$7/13\approx 0.538462$		3125	11	0.992671

Table 2. Table 2: The rate of profit μ ​ ( r , 0 , γ ​ A + ( 1 − γ ) ​ B ) 𝜇 𝑟 0 𝛾 𝐴 1 𝛾 𝐵 \mu(r,0,\gamma A+(1-\gamma)B) . Here, for a given odd r 𝑟 r , we choose γ 𝛾 \gamma to maximize γ ′ ↦ μ ​ ( r , 0 , γ ′ ​ A + ( 1 − γ ′ ) ​ B ) maps-to superscript 𝛾 ′ 𝜇 𝑟 0 superscript 𝛾 ′ 𝐴 1 superscript 𝛾 ′ 𝐵 \gamma^{\prime}\mapsto\mu(r,0,\gamma^{\prime}A+(1-\gamma^{\prime})B) . Results are rounded to six significant digits.

		$\mu (r,0,\gamma A$				$\mu (r,0,\gamma A$
$r$	$\gamma$	$+(1-\gamma )B)$		0$r$	$\gamma$	$+(1-\gamma )B)$
3	0.407641	0.133369		0025	0.2779260	0.482769
5	0.420756	0.229111		0125	0.1507220	0.709914
7	0.399201	0.279864		0625	0.0739646	0.854806
9	0.376138	0.318393		3125	0.0345306	0.931535

Equations

$$p_0=\frac{1}{10}\text{and}{p}_1=\frac{3}{4}.$$

$$\mu\big{(}{\textstyle\frac{1}{2}}A+{\textstyle\frac{1}{2}}B\big{)}=\frac{18}{709}\approx 0.0253879,$$

$$\mu (ABB)=\frac{2416}{35601}\approx \mathrm{0.0678633.}$$

$$\mu (ABABB)=\frac{3613392}{47747645}\approx \mathrm{0.0756769.}$$

$$(1-p_0)(1-p_1{)}^{r-1}=p_0{p}_1^{r-1},$$

$$p_0=\frac{{\rho }^{r-1}}{1+{\rho }^{r-1}}\text{and}{p}_1=\frac{1}{1+\rho }$$

$$\underset{r\ge 3,\rho \in (0,1),\Gamma (A,B)\text{ arbitrary}}{\sup }\mu (r,\rho ,\Gamma (A,B))=1.$$

$$\mu (3,\rho ,ABB)=\frac{(1-\rho {)}^3(1+\rho )(1+2\rho +{\rho }^2+2{\rho }^3+{\rho }^4)}{3+12\rho +20{\rho }^2+28{\rho }^3+36{\rho }^4+28{\rho }^5+20{\rho }^6+12{\rho }^7+3{\rho }^8}.$$

$$\mu (3,0,ABB)=\frac{1}{3}$$

$$\mu (3,0,ABABB)=\frac{9}{25}.$$

$$\Gamma (A,B)=(AB{)}^s{B}^{r-2}$$

$$\mu (r,0,(AB{)}^s{B}^{r-2})=\frac{r}{2s+r-2}\frac{2^s-1}{2^s+1},$$

$$\mu(r,0,(AB)^{s}B^{r-2})=\begin{cases}\cfrac{r}{2s+r-2}\;\displaystyle{\sum_{k=0}^{s}\bigg{\lceil}\frac{2k}{r}\bigg{\rceil}\binom{s}{k}\frac{1}{2^{s}}}&\text{if initial capital is even},\\ \vskip 6.0pt plus 2.0pt minus 2.0pt\cr 0&\text{if initial capital is odd}.\end{cases}$$

$$\mu :=t^{-1}\mathbf{\pi }({\dot{\mathbf{P}}}_{C_1}+{\mathbf{P}}_{C_1}{\dot{\mathbf{P}}}_{C_2}+\cdots +{\mathbf{P}}_{C_1}{\mathbf{P}}_{C_2}\cdots {\mathbf{P}}_{C_{t-1}}{\dot{\mathbf{P}}}_{C_t})1,$$

$$(X_0,X_1,\dots ,X_t),(X_t,X_{t+1},\dots ,X_{2t}),(X_{2t},X_{2t+1},\dots ,X_{3t}),\dots$$

$$\mathbf{\pi }(x_0){\mathbf{P}}_{C_1}(x_0,x_1){\mathbf{P}}_{C_2}(x_1,x_2)\cdots {\mathbf{P}}_{C_t}(x_{t-1},x_t)>0,$$

$$\mathbf{Q}((x_0,x_1,\dots ,x_t),(x_t,x_{t+1},\dots ,x_{2t}))$$

$$={\mathbf{P}}_{C_1}(x_t,x_{t+1}){\mathbf{P}}_{C_2}(x_{t+1},x_{t+2})\cdots {\mathbf{P}}_{C_t}(x_{2t-1},x_{2t}).$$

$${\mathbf{Q}}^n((x_0,x_1,\dots ,x_t),(y_0,y_1,\dots ,y_t))$$

$$={\mathbf{P}}^{n-1}(x_t,y_0){\mathbf{P}}_{C_1}(y_0,y_1){\mathbf{P}}_{C_2}(y_1,y_2)\cdots {\mathbf{P}}_{C_t}(y_{t-1},y_t).$$

$${\mathbf{P}}_A=\left(\begin{array}{ccc}0& 1/2& 1/2\\ 1/2& 0& 1/2\\ 1/2& 1/2& 0\end{array}\right),{\mathbf{P}}_B=\left(\begin{array}{ccc}0& 1/10& 9/10\\ 1/4& 0& 3/4\\ 3/4& 1/4& 0\end{array}\right),$$

$$\mathbf{W}=\left(\begin{array}{ccc}0& 1& -1\\ -1& 0& 1\\ 1& -1& 0\end{array}\right).$$

$$\Sigma =\{0,1,\dots ,r-1\}$$

$${\mathbf{P}}_A=\left(\begin{array}{ccccccc}0& 1/2& 0& \cdots & 0& 0& 1/2\\ 1/2& 0& 1/2& \cdots & 0& 0& 0\\ 0& 1/2& 0& \cdots & 0& 0& 0\\ ⋮& ⋮& ⋮& & ⋮& ⋮& ⋮\\ 0& 0& 0& \cdots & 0& 1/2& 0\\ 0& 0& 0& \cdots & 1/2& 0& 1/2\\ 1/2& 0& 0& \cdots & 0& 1/2& 0\end{array}\right),$$

$${\mathbf{P}}_B=\left(\begin{array}{ccccccc}0& p_0& 0& \cdots & 0& 0& 1-p_0\\ 1-p_1& 0& p_1& \cdots & 0& 0& 0\\ 0& 1-p_1& 0& \cdots & 0& 0& 0\\ ⋮& ⋮& ⋮& & ⋮& ⋮& ⋮\\ 0& 0& 0& \cdots & 0& p_1& 0\\ 0& 0& 0& \cdots & 1-p_1& 0& p_1\\ p_1& 0& 0& \cdots & 0& 1-p_1& 0\end{array}\right),$$

$$\mathbf{W}=\left(\begin{array}{ccccccc}0& 1& 0& \cdots & 0& 0& -1\\ -1& 0& 1& \cdots & 0& 0& 0\\ 0& -1& 0& \cdots & 0& 0& 0\\ ⋮& ⋮& ⋮& & ⋮& ⋮& ⋮\\ 0& 0& 0& \cdots & 0& 1& 0\\ 0& 0& 0& \cdots & -1& 0& 1\\ 1& 0& 0& \cdots & 0& -1& 0\end{array}\right).$$

$$\mathbf{Q}=\left(\begin{array}{cccc}1/2& 1/2& 0& 0\\ 0& 0& 1/2& 1/2\\ 1/2& 1/2& 0& 0\\ 1/2& 1/2& 0& 0\end{array}\right),$$

$${\mu }_j:=t^{-1}{\mathbf{\pi }}_j({\dot{\mathbf{P}}}_{C_1}+{\mathbf{P}}_{C_1}{\dot{\mathbf{P}}}_{C_2}+\cdots +{\mathbf{P}}_{C_1}{\mathbf{P}}_{C_2}\cdots {\mathbf{P}}_{C_{t-1}}{\dot{\mathbf{P}}}_{C_t})1$$

$$\displaystyle(2t)^{-1}\bm{\pi}_{1}\big{[}\dot{\bm{P}}_{C_{1}}+\bm{P}_{C_{1}}\dot{\bm{P}}_{C_{2}}+\cdots+\bm{P}_{C_{1}}\bm{P}_{C_{2}}\cdots\bm{P}_{C_{t-1}}\dot{\bm{P}}_{C_{t}}$$

$$\displaystyle\qquad\qquad\;{}+\bm{P}(\dot{\bm{P}}_{C_{1}}+\bm{P}_{C_{1}}\dot{\bm{P}}_{C_{2}}+\cdots+\bm{P}_{C_{1}}\bm{P}_{C_{2}}\cdots\bm{P}_{C_{t-1}}\dot{\bm{P}}_{C_{t}})\big{]}\bm{1}$$

$$=t^{-1}\mathbf{\pi }({\dot{\mathbf{P}}}_{C_1}+{\mathbf{P}}_{C_1}{\dot{\mathbf{P}}}_{C_2}+\cdots +{\mathbf{P}}_{C_1}{\mathbf{P}}_{C_2}\cdots {\mathbf{P}}_{C_{t-1}}{\dot{\mathbf{P}}}_{C_t})1,$$

$$\mu (4,\rho ,ABB)=\frac{(1-\rho {)}^3}{3(1+{\rho }^3)}$$

Full text

1 Introduction

The Parrondo effect appears when two fair coin-tossing games, $A$ and $B$, played in a random sequence or in some periodic sequence such as $ABB\,ABB\,ABB\,\cdots$, form a winning game. Let us define a $p$-coin to be a coin with probability $p$ of heads. In the original capital-dependent games of Parrondo (Harmer and Abbott, 1999), game $A$ uses a fair coin, while game $B$ uses two biased coins, a $p_{0}$-coin if capital is congruent to 0 (mod 3) and a $p_{1}$-coin otherwise, where

[TABLE]

(These coins can be physically realized with dice; see Figure 1.) The player wins one unit with heads and loses one unit with tails. Both games are fair, but the random mixture, denoted by $\frac{1}{2}A+\frac{1}{2}B$ and interpreted as the game in which the toss of a fair coin determines whether game $A$ or game $B$ is played, has long-term cumulative profit per game played (hereafter, rate of profit)

[TABLE]

and the pattern $ABB$ has rate of profit

[TABLE]

Dinis (2008) found that the pattern $ABABB$ has the highest rate of profit, namely

[TABLE]

These rates of profit are rather modest. Can we modify the games to make the rates of profit more substantial? To put it more precisely, how large can the rate of profit be if we vary the parameters of the games, subject to a fairness constraint? We will focus on periodic sequences, where the rates of profit tend to be larger than with random sequences.

Game $A$ is always the same fair-coin-tossing game. With $r\geq 3$ an integer, game $B$ is a mod $r$ capital-dependent game that uses two biased coins, a $p_{0}$-coin ($p_{0}<1/2$) if capital is congruent to 0 (mod $r$), and a $p_{1}$-coin ($p_{1}>1/2$) otherwise. The probabilities $p_{0}$ and $p_{1}$ must be such that game $B$ is fair, which requires the constraint

[TABLE]

or equivalently,

[TABLE]

for some $\rho\in(0,1)$. The special case of $r=3$ and $\rho=1/3$ gives (1). The games are played in some pattern $\Gamma(A,B)$, repeated ad infinitum. We denote the rate of profit by $\mu(r,\rho,\Gamma(A,B))$, so that the rates of profit in (2) and (3) in this notation become $\mu(3,1/3,ABB)$ and $\mu(3,1/3,ABABB)$.

How large can $\mu(r,\rho,\Gamma(A,B))$ be? The answer, perhaps surprisingly, is that it can be arbitrarily close to 1 (i.e., 100%).

Theorem 1**.**

[TABLE]

The proof is deferred to Section 4.

We can compute $\mu(r,\rho,\Gamma(A,B))$ for $r\geq 3$ (the modulo number in game $B$) and pattern $\Gamma(A,B)$ as a function of $\rho$ (the parameter in (4)). Indeed, the method of Ethier and Lee (2009) applies if $r$ is odd, and generalizations of it apply if $r$ is even; see Section 2 for details. For example,

[TABLE]

This and other examples suggest that typically $\mu(r,\rho,\Gamma(A,B))$ is decreasing in $\rho$, hence maximized at $\rho=0$. (There are exceptions, which include, when $r\geq 3$ is odd, $AB^{s}$ with $s\geq 3$ odd.) We excluded the case $\rho=0$ in (4), but now we want to include it. We find that

[TABLE]

(by (5)) and

[TABLE]

Thus, we take $\rho=0$ in what follows.

For a given $r\geq 3$, we expect that we can maximize the rate of profit $\mu(r,0,\Gamma(A,B))$ with a pattern of the form

[TABLE]

for some positive integer $s$. Notice that this is $ABB$ if $(r,s)=(3,1)$ and $ABABB$ if $(r,s)=(3,2)$.

Let us explain the intuition behind (8). Only the $s$ plays of game $A$ are random. Game $B$ is deterministic and very simple: If capital is congruent to 0 (mod $r$), we lose one unit, otherwise we win one unit. Notice that cumulative profit remains bounded by $r$ when game $B$ is played repeatedly, hence cumulative profit per game played tends to 0 as the number of games played tends to infinity, and game $B$ is (asymptotically) fair.

Clearly, the optimal strategy, if it were legal, would be to play game $A$ when capital is congruent to 0 (mod $r$) and to play game $B$ otherwise. With initial capital congruent to 0 (mod $r$), this strategy could be described as playing the pattern $(AB)^{S}B^{r-2}$, where $S$ is the geometric random variable equal to the number of plays of game $A$ needed to achieve a win at that game. Of course, random patterns are not ordinarily considered, so (8) seems a reasonable nonrandom approximation for some positive integer $s$.

First, assume that $r$ is odd and initial capital is congruent to 0 (mod $r$). If all $s$ plays of game $A$ result in losses, cumulative profit is $-1$ after one play of (8); otherwise it is $r$. If initial capital is congruent to $r-1$ (mod $r$), then after one play of (8), cumulative profit is 1 with probability 1.

Second, assume that $r$ is even and again initial capital is congruent to 0 (mod $r$). If the number of wins in the $s$ plays of game $A$ is 0, cumulative profit is [math] after one play of (8); if the number of wins is between 1 and $r/2$, inclusive, cumulative profit is $r$; if the number of wins is between $r/2+1$ and $r$, inclusive, cumulative profit is $2r$; if the number of wins is between $r+1$ and $3r/2$, inclusive, cumulative profit is $3r$; and so on. If initial capital is congruent to $r-1$ (mod $r$), then after one play of (8), cumulative profit is 0 with probability 1.

The probabilistic structure of capital growth after multiple plays of (8) can be analyzed precisely from these observations, and we can evaluate the exact rate of profit.

Theorem 2**.**

Let $r\geq 3$ be an odd integer and $s$ be a positive integer. Then

[TABLE]

regardless of initial capital.

Let $r\geq 4$ be an even integer and $s$ be a positive integer. Then

[TABLE]

The formula in (9) is consistent with (6) and (7). The sum in (10) is equal to $(2^{s}-1)/2^{s}$ if $s\leq r/2$ and bounded below by $(2^{s}-1)/2^{s}$ in general. Theorem 2 implies Theorem 1, as we will confirm later. The proof of Theorem 2 is deferred to Section 4. Table 1 illustrates (9) with several examples.

We do not consider random mixtures $\gamma A+(1-\gamma)B$ of games $A$ and $B$. Although we expect that the rate of profit, which we denote by $\mu(r,\rho,\gamma A+(1-\gamma)B)$, can be made arbitrarily close to 1 by suitable choice of the modulo number $r$ in game $B$, the parameter $\rho$ in (4), and the probability $\gamma$ with which game $A$ is played, we cannot prove it. However, see Table 2 for several examples.

2 SLLN for periodic sequences of games

Ethier and Lee (2009) proved a strong law of large numbers and a central limit theorem for periodic sequences of Parrondo games of the form $A^{r}B^{s}$, repeated ad infinitum, where $r$ and $s$ are positive integers. Below we state a generalization of the SLLN to arbitrary patterns. Later we will weaken the hypotheses as needed.

First, it should be mentioned that several other authors have studied periodic sequences of Parrondo games. Pyke (2003) discussed one example, $AABB$, which he regarded as the alternation of $AA$ and $BB$. His method is sound but his stated “asymptotic average gain” for that example is inaccurate, and the source of the error is unknown. Kay and Johnson (2003) studied patterns of the form $A^{r}B^{s}$ in the context of history-dependent Parrondo games, and gave an expression for the rate of profit that is consistent with (11) below. Key, Kłosek, and Abbott (2006), as well as Rémillard and Vaillancourt (2019), took a different approach, analyzing periodic sequences of Parrondo games in terms of transience to $\pm\infty$ and recurrence instead of in terms of the rate of profit.

Theorem 3**.**

Let $\bm{P}_{A}$ and $\bm{P}_{B}$ be transition matrices for Markov chains in a finite state space $\Sigma$. Let $C_{1}C_{2}\cdots C_{t}$, where each $C_{i}$ is $A$ or $B$, be a pattern of $A$s and $B$s of length $t$. Assume that $\bm{P}:=\bm{P}_{C_{1}}\bm{P}_{C_{2}}\cdots\bm{P}_{C_{t}}$ is irreducible and aperiodic, and let the row vector $\bm{\pi}$ be the unique stationary distribution of $\bm{P}$. Given a real-valued function $w$ on $\Sigma\times\Sigma$, define the payoff matrix $\bm{W}:=(w(i,j))_{i,j\in\Sigma}$. Define $\dot{\bm{P}}_{A}:=\bm{P}_{A}\circ\bm{W}$ and $\dot{\bm{P}}_{B}:=\bm{P}_{B}\circ\bm{W}$, where $\circ$ denotes the Hadamard $($entrywise$)$ product, and put

[TABLE]

where $\bm{1}$ denotes a column vector of $1$s with entries indexed by $\Sigma$. Let $\{X_{n}\}_{n\geq 0}$ be a nonhomogeneous Markov chain in $\Sigma$ with transition matrices $\bm{P}_{C_{1}}$, $\bm{P}_{C_{2}}$, …, $\bm{P}_{C_{t}}$, $\bm{P}_{C_{1}}$, $\bm{P}_{C_{2}}$, …, $\bm{P}_{C_{t}}$, $\bm{P}_{C_{1}}$, and so on, and let the initial distribution be arbitrary. For each $n\geq 1$, define $\xi_{n}:=w(X_{n-1},X_{n})$ and $S_{n}:=\xi_{1}+\cdots+\xi_{n}$. Then $\lim_{n\to\infty}n^{-1}S_{n}=\mu$ a.s.

Proof.

The proof is identical to the proof of Theorem 6 of Ethier and Lee (2009). However, here we have assumed fewer hypotheses and should explain why. First, it is unnecessary to assume that $\bm{P}_{A}$ and $\bm{P}_{B}$ are irreducible and aperiodic because that assumption is not needed. It is also unnecessary to assume that all cyclic permutations of $\bm{P}:=\bm{P}_{C_{1}}\bm{P}_{C_{2}}\cdots\bm{P}_{C_{t}}$ are irreducible and aperiodic because that assumption is redundant; it suffices that $\bm{P}$ itself be irreducible and aperiodic. Finally, we assumed in the original theorem that the Markov chain

[TABLE]

is irreducible and aperiodic, and we claim that this assumption is also redundant. The state space $\Sigma^{*}$ of (12) is the set of $(x_{0},x_{1},\ldots,x_{t})\in\Sigma^{t+1}$ such that

[TABLE]

and its transition matrix $\bm{Q}$ is given by

[TABLE]

We use the fact that a necessary and sufficient condition for a finite Markov chain to be irreducible and aperiodic is that some power of its transition matrix has all entries positive. It is straightforward to show that $\bm{Q}^{n}$ has all entries positive if $\bm{P}^{n-1}$ does. Indeed,

[TABLE]

Because $\bm{P}$ is irreducible and aperiodic, so too is $\bm{Q}$. ∎

As an illustration, we can use (11) to confirm (2) and (3), in which case $\Sigma=\{0,1,2\}$,

[TABLE]

and the payoff matrix is

[TABLE]

More generally, we wish to apply Theorem 3 with

[TABLE]

($r$ is the modulo number in game $B$), the $r\times r$ transition matrices

[TABLE]

[TABLE]

where $p_{0}$ and $p_{1}$ are given by (4), and the $r\times r$ payoff matrix

[TABLE]

There are five cases that we want to consider.

Let the pattern $C_{1}C_{2}\cdots C_{t}$ of Theorem 3 be arbitrary. If $\rho>0$ and $r$ is odd ($\geq 3$), then $\bm{P}:=\bm{P}_{C_{1}}\bm{P}_{C_{2}}\cdots\bm{P}_{C_{t}}$ is irreducible and aperiodic. 2. 2.

Let the pattern $C_{1}C_{2}\cdots C_{t}$ be arbitrary. If $\rho>0$, $r$ is even ($\geq 4$), and $t$ is odd, then $\bm{P}$ is irreducible and periodic with period 2. 3. 3.

Let the pattern $C_{1}C_{2}\cdots C_{t}$ be arbitrary. If $\rho>0$, $r$ is even ($\geq 4$), and $t$ is even, then $\bm{P}$ is reducible with two aperiodic recurrent classes, each of size $r/2$. 4. 4.

Let the pattern $C_{1}C_{2}\cdots C_{t}$ have the form $(AB)^{s}B^{r-2}$ for a positive integer $s$. If $\rho=0$ and $r$ is odd ($\geq 3$), then $\bm{P}:=(\bm{P}_{A}\bm{P}_{B})^{s}(\bm{P}_{B})^{r-2}$ is reducible with one aperiodic recurrent class of size 2 and $r-2$ transient states. 5. 5.

Let the pattern $C_{1}C_{2}\cdots C_{t}$ have the form $(AB)^{s}B^{r-2}$ for a positive integer $s$. If $\rho=0$ and $r$ is even ($\geq 4$), then $\bm{P}$ is reducible with two absorbing states and $r-2$ transient states.

Theorem 3 applies directly only to Case 1. Nevertheless, the theorem can be extended so as to apply first to Cases 1 and 4, then to Cases 3 and 5, and finally to Case 2. We begin by generalizing Theorem 3 so as to apply to Cases 1 and 4.

Theorem 3*′*.

Theorem 3 holds with “is irreducible and aperiodic” replaced by “has only one recurrent class, which is aperiodic”.

Proof.

Assume that $\bm{P}$ has only one recurrent class, which is aperiodic. Let $\Sigma_{0}\subset\Sigma$ be the unique recurrent class. The stationary distribution $\bm{\pi}$ of $\bm{P}$ is unique and satisfies $\bm{\pi}(x)>0$ if $x\in\Sigma_{0}$ and $\bm{\pi}(x)=0$ otherwise. For some $n\geq 2$, $\bm{P}^{n-1}(x_{t},y_{0})>0$ for all $x_{t},y_{0}\in\Sigma_{0}$. With the help of (2) we find that $\bm{Q}^{n}$ has all entries positive, hence $\bm{Q}$ is irreducible and aperiodic.

An example may help to clarify this argument. Consider the special case of (14)–(17) (with (4)) in which $\rho=0$ and $r=3$, and let $C_{1}C_{2}C_{3}=ABB$. Then $\bm{\pi}=(2/3,0,1/3)$, and the state space for the Markov chain $(X_{0},X_{1},X_{2},X_{3})$, $(X_{3},X_{4},X_{5},X_{6})$, … is $\Sigma^{*}=\{(0,1,2,0),(0,2,0,2),(2,0,2,0),(2,1,2,0)\}$ with corresponding transition matrix

[TABLE]

which is irreducible and aperiodic.

The remainder of the proof follows that of Theorem 6 of Ethier and Lee (2009). ∎

We turn to Cases 3 and 5, which require a new formulation of Theorem 3, the difficulty being that the limit in the SLLN depends on the initial distribution of the underlying Markov chain.

Theorem 4**.**

Let $\bm{P}_{A}$ and $\bm{P}_{B}$ be transition matrices for Markov chains in a finite state space $\Sigma$. Let $C_{1}C_{2}\cdots C_{t}$, where each $C_{i}$ is $A$ or $B$, be a pattern of $A$s and $B$s of length $t$. Assume that $\bm{P}:=\bm{P}_{C_{1}}\bm{P}_{C_{2}}\cdots\bm{P}_{C_{t}}$ is reducible with two recurrent classes $R_{1}$ and $R_{2}$, both of which are aperiodic, and possibly some transient states, and let the row vectors $\bm{\pi}_{1}$ and $\bm{\pi}_{2}$ be the unique stationary distributions of $\bm{P}$ concentrated on $R_{1}$ and $R_{2}$, respectively. Given a real-valued function $w$ on $\Sigma\times\Sigma$, define the payoff matrix $\bm{W}:=(w(i,j))_{i,j\in\Sigma}$. Define $\dot{\bm{P}}_{A}:=\bm{P}_{A}\circ\bm{W}$ and $\dot{\bm{P}}_{B}:=\bm{P}_{B}\circ\bm{W}$, where $\circ$ denotes the Hadamard $($entrywise$)$ product, and put

[TABLE]

for $j=1,2$, where $\bm{1}$ denotes a column vector of $1$s with entries indexed by $\Sigma$. Let $\{X_{n}\}_{n\geq 0}$ be a nonhomogeneous Markov chain in $\Sigma$ with transition matrices $\bm{P}_{C_{1}}$, $\bm{P}_{C_{2}}$, …, $\bm{P}_{C_{t}}$, $\bm{P}_{C_{1}}$, $\bm{P}_{C_{2}}$, …, $\bm{P}_{C_{t}}$, $\bm{P}_{C_{1}}$, and so on, and let its initial state be $i_{0}\in\Sigma$. Let $\alpha:=P(X_{nt}\in R_{1}\text{ for }n\text{ sufficiently large})$. For each $n\geq 1$, define $\xi_{n}:=w(X_{n-1},X_{n})$ and $S_{n}:=\xi_{1}+\cdots+\xi_{n}$. Then $\lim_{n\to\infty}n^{-1}S_{n}=\alpha\mu_{1}+(1-\alpha)\mu_{2}$ a.s.

Proof.

The argument used to prove the conclusion of Theorem 3 when $\bm{\pi}$ is the initial distribution applies here, allowing us to prove that $\lim_{n\to\infty}n^{-1}S_{n}=\mu_{j}$ a.s. if $\bm{\pi}_{j}$ is the initial distribution, then if the initial state $i_{0}$ belongs to $R_{j}$, for $j=1,2$. Let $N:=\min\{nt:X_{nt}\in R_{1}\cup R_{2}\}$. Then $P(X_{N}\in R_{1})=\alpha$, and the stated conclusion readily follows. ∎

We conclude this section by addressing Case 2.

Theorem 3*′′*.

Theorem 3 holds with “is irreducible and aperiodic” replaced by “is irreducible and periodic with period 2”.

Proof.

The idea is to apply Theorem 4 with the pattern $C_{1}C_{2}\cdots C_{t}$ replaced by the pattern $C_{1}C_{2}\cdots C_{t}C_{1}C_{2}\cdots C_{t}$, which has the same limit in the SLLN. In particular, $\bm{P}$ is replaced by $\bm{P}^{2}$. The assumption that $\bm{P}$ is irreducible with period 2 means that $\Sigma$ is the disjoint union of $R_{1}$ and $R_{2}$, and transitions under $\bm{P}$ take $R_{1}$ to $R_{2}$ and $R_{2}$ to $R_{1}$. This means that $\bm{P}^{2}$ is reducible with two recurrent classes, $R_{1}$ and $R_{2}$, and no transient states. Let the row vectors $\bm{\pi}_{1}$ and $\bm{\pi}_{2}$ be the unique stationary distributions of $\bm{P}^{2}$ concentrated on $R_{1}$ and $R_{2}$, respectively. Then $\bm{\pi}_{1}\bm{P}=\bm{\pi}_{2}$ and $\bm{\pi}_{2}\bm{P}=\bm{\pi}_{1}$. Consequently, the limit $\mu_{1}$ starting in $R_{1}$ is, according to Theorem 4,

[TABLE]

where $\bm{\pi}:=(\bm{\pi}_{1}+\bm{\pi}_{2})/2$ is the unique stationary distribution of $\bm{P}$, and this is (11). The limit $\mu_{2}$ starting in $R_{2}$ is the same but with $\bm{\pi}_{1}$ and $\bm{\pi}_{2}$ interchanged, and again this is (11). ∎

For example, we find that

[TABLE]

as a consequence of Theorem $3^{\prime\prime}$, and

[TABLE]

as a consequence of Theorem 4. Recalling the five cases below (14)–(17), these two examples correspond to Cases 2 and 3, respectively, whereas (5) corresponds to Case 1.

Finally, we point out that Rémillard and Vaillancourt (2019) have addressed some of the same issues that we encountered in this section, namely reducibility, periodicity, and more than one recurrent class, albeit by different methods.

3 Mean of a binomial-like distribution

Here we want to find the mean of a discrete distribution that depends, like the binomial, on two parameters, a positive integer $n$ and $p\in(0,1)$. The distribution does not appear to have a name. The formula for the probability mass function depends on whether $n$ is even or odd, so we treat the two cases separately. We use the convention that $q:=1-p$.

In the case $n=2m$ with $m$ a positive integer, consider a particle that starts at $(0,0)$. At each time step, it moves one unit to the right with probability $p$ or one unit up with probability $q$, stopping at the first time it reaches the boundary $(k,m-\lfloor k/2\rfloor)$, $k=0,1,\ldots,2m$. Let $Z_{2m}$ denote the $x$-coordinate of its final position. Then

[TABLE]

Each lattice path ending at $(k,m-\lfloor k/2\rfloor)$ has probability $p^{k}q^{m-\lfloor k/2\rfloor}$, and the binomial coefficient counts the number of paths that end at $(k,m-k/2)$ if $k$ is even, and at $(k,m-(k-1)/2)$ if $k$ is odd because in the latter case the path must first reach $(k,m-(k+1)/2)$. See Figure 2.

In the case $n=2m-1$ with $m$ a positive integer, again consider a particle that starts at $(0,0)$. At each time step, it moves one unit to the right with probability $p$ or one unit up with probability $q$, stopping at the first time it reaches the boundary $(k,m-\lceil k/2\rceil)$, $k=0,1,\ldots,2m-1$. Let $Z_{2m-1}$ denote the $x$-coordinate of its final position. Then

[TABLE]

Each lattice path ending at $(k,m-\lceil k/2\rceil)$ has probability $p^{k}q^{m-\lceil k/2\rceil}$, and the binomial coefficient counts the number of paths that end at $(k,m-(k+1)/2)$ if $k$ is odd, and at $(k,m-k/2)$ if $k$ is even because in the latter case the path must first reach $(k,m-1-k/2)$. See Figure 3.

Lemma 5**.**

[TABLE]

Equivalently,

[TABLE]

Proof.

We give separate proofs for $n$ even and $n$ odd, both by induction. To initialize, in the $n=1$ case, the probability mass function is $q$ at 0 and $p$ at 1, so (20) holds. In the $n=2$ case, the probability mass function is $q$ at 0, $pq$ at 1, and $p^{2}$ at 2, so again (20) holds.

Now assume that (20) holds for $n=2m$. We must show that it holds for $n=2m+2$. By the interpretation of the distribution (see Figure 2),

[TABLE]

We conclude that

[TABLE]

proving the lemma when $n$ is even.

Now assume that (20) holds for $n=2m-1$. We must show that it holds for $n=2m+1$. By the interpretation of the distribution (see Figure 3),

[TABLE]

We conclude that

[TABLE]

proving the lemma when $n$ is odd. ∎

Lemma 6**.**

[TABLE]

Equivalently,

[TABLE]

Proof.

As with Lemma 5, we give separate proofs for $n$ even and $n$ odd, both by induction. To initialize, in the $n=1$ case, the probability mass function is $q$ at 0 and $p$ at 1, so the mean is $p=1-q$ and (21) holds. In the $n=2$ case, the probability mass function is $q$ at 0, $pq$ at 1, and $p^{2}$ at 2, so the mean is $pq+2p^{2}=(1-q)(2-q)$ and again (21) holds.

Now assume that (21) holds for $n=2m$. We must show that it holds for $n=2m+2$. By the interpretation of the distribution (see Figure 2),

[TABLE]

We conclude from the induction hypothesis and Lemma 5 that

[TABLE]

proving the lemma when $n$ is even.

Now assume that (21) holds for $n=2m-1$. We must show that it holds for $n=2m+1$. By the interpretation of the distribution (see Figure 3),

[TABLE]

We conclude from the induction hypothesis and Lemma 5 that

[TABLE]

proving the lemma when $n$ is odd. ∎

We conclude this section with alternative interpretations of the distribution of $Z_{n}$, given by (18) if $n=2m$ and by (19) if $n=2m-1$, that do not require separate formulations for $n$ even and $n$ odd.

• •

Consider a particle that starts at $(0,0)$. At each time step, it moves one unit to the right with probability $p$ or one unit up with probability $q$, stopping at the first time it reaches or crosses the boundary $(k,(n-k)/2)$, $k=0,1,\ldots,n$. Let $Z_{n}$ denote the $x$-coordinate of its final position.

• •

Consider a particle that starts at $(0,0)$. At each time step, it moves one unit to the right with probability $p$ or two units up with probability $q$, stopping at the first time it reaches or crosses the boundary $(k,n-k)$, $k=0,1,\ldots,n$. Let $Z_{n}$ denote the $x$-coordinate of its final position.

• •

Consider a particle that starts at $(0,0)$. At each time step, it moves one unit to the right with probability $p$ or one unit up with probability $q$ followed by another unit up with probability 1, stopping at the first time it reaches the boundary $(k,n-k)$, $k=0,1,\ldots,n$. Let $Z_{n}$ denote the $x$-coordinate of its final position.

The last of these interpretations is the context in which the distribution arises in Section 4 below.

4 Proofs of Theorems 1 and 2

Proof of Theorem 1.

The result is immediate from Theorem 2 provided we can show that $f(\rho):=\mu(r,\rho,(AB)^{s}B^{r-2})$ is continuous at 0. We use Theorem 3, $3^{\prime\prime}$, or 4 to evaluate $f(\rho)$, which is a rational function of $\rho$. The only potential singularities are those of the stationary distribution $\bm{\pi}$ (or $\bm{\pi}_{1}$ or $\bm{\pi}_{2}$). But the existence and uniqueness of $\bm{\pi}$ (or $\bm{\pi}_{1}$ or $\bm{\pi}_{2}$) for $0\leq\rho\leq 1$ ensures that $f(\rho)$ is real analytic there, hence continuous. ∎

We give two proofs of Theorem 2, the first one direct (depending solely on Theorems $3^{\prime}$ and 4) but complicated, and the second one more easily understood but depending on Theorems $3^{\prime}$ and 4 and Lemmas 5 and 6.

First proof of Theorem 2.

First assume that $r\geq 3$ is odd. Since $\dot{\bm{P}}_{A}\bm{1}=\bm{0}$, Theorem $3^{\prime}$ tells us that the rate of profit, regardless of initial capital, can be expressed as

[TABLE]

where $\bm{\pi}$ is the stationary distribution of $\bm{P}:=(\bm{P}_{A}\bm{P}_{B})^{s}({\bm{P}}_{B})^{r-2}$. Since $\rho=0$ and $r$ is odd, $\bm{P}$ is reducible with one recurrent class $\{0,r-1\}$ and $r-2$ transient states. From the observations about the pattern $(AB)^{s}B^{r-2}$ in Section 1 it follows that $\bm{P}(0,0)=1-\bm{P}(0,r-1)=1-2^{-s}$ and $\bm{P}(r-1,0)=1$ so that the stationary distribution $\bm{\pi}$ is given by $\bm{\pi}=(\pi_{0},0,0,\ldots,0,\pi_{r-1})$, where

[TABLE]

Except for the factor $(2s+r-2)^{-1}$, all of the terms in (4) have the form $\bm{\pi}\bm{\Lambda}\dot{\bm{P}}_{B}\bm{1}$ for a transition matrix $\bm{\Lambda}=(\lambda_{i,j})_{i,j=0,1,\ldots,r-1}$. Therefore, using $\dot{\bm{P}}_{B}\bm{1}=(-1,1,1,\ldots,1)^{\textsf{T}}$, we have

[TABLE]

showing that we need only determine two of the entries of $\bm{\Lambda}$ to evaluate (23).

We first consider the transition matrix $\bm{\Lambda}=(\bm{P}_{A}\bm{P}_{B})^{j}\bm{P}_{A}$ for $0\leq j\leq s-1$. When $j<(r-1)/2$, we have $\lambda_{0,0}=0$. When $j\geq(r-1)/2$, from state 0 we can reach state $r-1$ after $j$ plays of $AB$ if there are at least $(r-1)/2$ wins from the $j$ plays of game $A$, after which we can move to state 0 with an additional win from game $A$. Thus, we have

[TABLE]

For all $j$, we have $\lambda_{r-1,0}=1/2.$ Using (23), for $j<(r-1)/2$,

[TABLE]

and for $j\geq(r-1)/2$,

[TABLE]

Summing these $s$ terms, we have

[TABLE]

Next we consider the transition matrix $\bm{\Lambda}=(\bm{P}_{A}\bm{P}_{B})^{s}(\bm{P}_{B})^{i}$ for $0\leq i\leq r-3$. For even $i$, we have $\lambda_{0,0}=2^{-s}$ and $\lambda_{r-1,0}=0$, from which we obtain, via (23),

[TABLE]

Now let $i$ be odd. Assume we start from state 0. With at least $(r-i)/2$ wins from $s$ plays of game $A$, we can reach state $r-i$ or an even state to its right after $s$ plays of game $AB$, and then move to state 0 after $i$ additional plays of game $B$. Thus, we have

[TABLE]

Moreover, $\lambda_{r-1,0}=1$. Thus, for odd $i$ we obtain, via (23),

[TABLE]

Summing over $i$, we have

[TABLE]

For the double sum in (25), a change of variables gives

[TABLE]

There are two cases. If $(r-1)/2\geq s$, which also makes the double sum in (24) zero, then this becomes

[TABLE]

and (4) becomes

[TABLE]

If $(r-1)/2<s$, it suffices to verify the following identity:

[TABLE]

For $1\leq s_{0}<s$,

[TABLE]

where the first equality uses (4) and the last equality uses the relationship between the binomial and negative binomial distributions. (The first sum within brackets is the probability that, in a sequence of independent Bernoulli trials with success probability $1/2$, at most $s$ trials are needed for the $k$th success, and the second sum is the probability that at least $k$ successes occur in $s$ trials.)

Next assume that $r\geq 4$ is even. Theorem 4 tells us that the rate of profit can be expressed as

[TABLE]

where $\bm{\pi}_{0}:=(1,0,0,\ldots,0)$ if initial capital is even and $\bm{\pi}_{0}:=(0,0,\ldots,0,1)$ if initial capital is odd.

Except for the factor $(2s+r-2)^{-1}$, all of the terms in (4) have the form $\bm{\pi}_{0}\bm{\Lambda}\dot{\bm{P}}_{B}\bm{1}$ for a transition matrix $\bm{\Lambda}=(\lambda_{i,j})_{i,j=0,1,\ldots,r-1}$, and

[TABLE]

For $\bm{\Lambda}=(\bm{P}_{A}\bm{P}_{B})^{j}\bm{P}_{A}$ with $0\leq j\leq s-1$, $\lambda_{0,0}=0$ and $\lambda_{r-1,0}=1/2$. For $\bm{\Lambda}=(\bm{P}_{A}\bm{P}_{B})^{s}(\bm{P}_{B})^{i}$ with $0\leq i\leq r-3$, $\lambda_{0,0}=0$ if $i$ is odd and

[TABLE]

if $i$ is even. Finally, $\lambda_{r-1,0}=1$ if $i$ is odd and $\lambda_{r-1,0}=0$ if $i$ is even.

Therefore, if initial capital is odd,

[TABLE]

and if initial capital is even,

[TABLE]

It remains to check that this last expression coincides with the formula in (10). The quantity within braces is equal to

[TABLE]

and the proof is complete. ∎

Second proof of Theorem 2.

First, fix an odd integer $r\geq 3$ and a positive integer $s$. We apply Theorem $3^{\prime}$ assuming (14)–(17) with $\rho=0$ in (4) and $C_{1}C_{2}\cdots C_{t}=(AB)^{s}B^{r-2}$ with $t:=2s+r-2$, to conclude that

[TABLE]

(The theorem tells us that the rate of profit does not depend on initial capital, so for convenience we take initial capital congruent to 0 (mod $r$).) Here $S_{1},S_{2},\ldots$ is the player’s sequence of cumulative profits. We can evaluate $E[S_{nt}]$.

We denote by $p_{n}(k)$, $k=0,1,\ldots,n$, the probability mass function in (18) if $n=2m$ and in (19) if $n=2m-1$. We claim that

[TABLE]

with $p=1-2^{-s}$. The result follows by using the third of the alternative interpretations of the distribution in (18) and (19) at the end of Section 3.

We can now evaluate, with the help of Lemmas 5 and 6, mean profit after $nt$ games:

[TABLE]

We divide by $nt=n(2s+r-2)$ and let $n\to\infty$ to obtain

[TABLE]

so (9) follows from this and (28).

Second, fix an even integer $r\geq 4$ and a positive integer $s$. We apply Theorem 4 assuming (14)–(17) with $\rho=0$ in (4) and $C_{1}C_{2}\cdots C_{t}=(AB)^{s}B^{r-2}$ with $t:=2s+r-2$, to conclude that (28) holds. (The theorem tells us that the rate of profit depends on initial capital only through its parity, so for convenience we take initial capital congruent to 0 (mod $r$) if initial capital is even, or congruent to $r-1$ (mod $r$) if odd.) Recalling from Section 1 that, with initial capital congruent to 0 (mod $r$), each play of $(AB)^{s}B^{r-2}$ results in a mean profit of

[TABLE]

we find that

[TABLE]

With initial capital congruent to $r-1$ (mod $r$), $P(S_{nt}=0)=1$, so

[TABLE]

and (10) follows from the last two limits and (28). ∎

Acknowledgments

We are grateful to Derek Abbott for raising the question addressed here and to Ira Gessel for suggesting the lattice path interpretation of the distribution defined by (18) and (19).