Concerning an adversarial version of the Last-Success-Problem

TL;DR

This paper analyzes an adversarial variation of the Last-Success-Problem involving two players who sequentially observe Bernoulli variables and decide whether to continue or pass, determining optimal strategies and winning probabilities.

Contribution

It introduces an adversarial version of the Last-Success-Problem, deriving optimal strategies and characterizing winning probabilities under different parameter settings.

Findings

Optimal strategies for both players are established.

Probability of Player A winning decreases with increasing n for uniform parameters.

As n grows, Player A's winning probability approaches approximately 0.4323.

Abstract

There are $n$ independent Bernoulli random variables with parameters $p_i$ that are observed sequentially. Two players, A and B, act in turns starting with player A. Each player has the possibility on his turn, when $I_k=1$, to choose whether to continue with his turn or to pass his turn on to his opponent for observation of the variable $I_{k+1}$. If $I_k=0$, the player must necessarily to continue with his turn. After observing the last variable, the player whose turn it is wins if $I_n=1$, and loses otherwise. We determine the optimal strategy for the player whose turn it is and establish the necessary and sufficient condition for player A to have a greater probability of winning than player B. We find that, in the case of $n$ Bernoulli random variables with parameters $1/n$, the probability of player A winning is decreasing with $n$ towards its limit $\frac{1}{2} - \frac{1}{2\,e^2}=0.4323323...$. We also study the game when the parameters are the results of uniform random variables, $\mathbf{U}[0,1]$

Equations

$$\mathbf{s}=\{\begin{array}{ll}\max \{1\le k\le n:{\sum }_{j=k}^n{r}_j\ge 1\},& \text{if ∑i=1nri≥1 };\\ 1,& \text{ otherwise }\end{array}$$

$$\mathcal{V}(p_1,...,p_n):=\left(\prod _{j=\mathbf{s}}^n{q}_j\right)\left(\sum _{i=\mathbf{s}}^n{r}_i\right)$$

$${\mathbf{V}}_k=p_k\cdot \max \{{\mathbf{V}}_{k+1},1-{\mathbf{V}}_{k+1}\}+(1-p_k)\cdot {\mathbf{V}}_{k+1};{\mathbf{V}}_n=p_n.$$

$$p_n={\mathbf{V}}_n<{\mathbf{V}}_{k+1}<{\mathbf{V}}_k<\frac{1}{2}.$$

$${\mathbf{V}}_k=p_k(1-{\mathbf{V}}_{k+1})+(1-p_k){\mathbf{V}}_{k+1}>p_k{\mathbf{V}}_{k+1}+(1-p_k){\mathbf{V}}_{k+1}={\mathbf{V}}_{k+1}.$$

$$\text{if }x,y\in [0,1/2)\text{ then }x(1-y)+(1-x)y<\frac{1}{2},$$

$${\mathbf{V}}_k=p_k(1-{\mathbf{V}}_{k+1})+(1-p_k){\mathbf{V}}_{k+1}<\frac{1}{2}.$$

$${\mathbf{V}}_1=...={\mathbf{V}}_{r-1}={\mathbf{V}}_r>\frac{1}{2}.$$

$$\text{If }x\in (1/2,1]\text{ and }y\in [0,1/2)\text{ then }x(1-y)+(1-x)y>\frac{1}{2}.$$

$${\mathbf{V}}_r=p_r\cdot \max \{{\mathbf{V}}_{r+1},1-{\mathbf{V}}_{r+1}\}+(1-p_r)\cdot {\mathbf{V}}_{r+1}$$

$$=p_r\cdot (1-{\mathbf{V}}_{r+1})+(1-p_r)\cdot {\mathbf{V}}_{r+1}>\frac{1}{2}$$

$${\mathbf{V}}_1=...={\mathbf{V}}_{r-1}={\mathbf{V}}_r.$$

$${\mathbf{V}}_1=...={\mathbf{V}}_{r-1}={\mathbf{V}}_r=\frac{1}{2}.$$

$${\mathbf{V}}_r=\frac{1}{2}\cdot \max \{{\mathbf{V}}_{r+1},1-{\mathbf{V}}_{r+1}\}+(1-\frac{1}{2})\cdot {\mathbf{V}}_{r+1}=\frac{1}{2}$$

$${\mathbf{V}}_1=...={\mathbf{V}}_{r-1}={\mathbf{V}}_r.$$

$${\mathfrak{u}}_r:=\{\begin{array}{ll}\max \Omega ,& \text{if Ωr=∅ };\\ r,& \text{if Ωr=∅. }\end{array}$$

$${\mathbf{V}}_1=P\left(\sum _{i=\mathfrak{u}}^n{I}_i=odd\right)$$

$${\mathbf{V}}_1>\frac{1}{e}.$$

$${\mathbf{V}}_1=\frac{1-{\left(1-2p\right)}^n}{2}<\frac{1}{2}.$$

$${\mathbf{V}}_1=\sum _{i=0}^{\frac{n}{2}-1}{\left(1-p\right)}^{-1-2i+n}{p}^{1+2i}\left(\genfrac{}{}{0ex}{}{n}{1+2i}\right)=\frac{1-{\left(1-2p\right)}^n}{2}.$$

$${\mathbf{V}}_1=\sum _{i=0}^{\frac{n-1}{2}}{\left(1-p\right)}^{-1-2i+n}{p}^{1+2i}\left(\genfrac{}{}{0ex}{}{n}{1+2i}\right)=\frac{1-{\left(1-2p\right)}^n}{2}.$$

$$\frac{1}{2}-\frac{1}{2e^2}=\mathrm{0.4323323...}$$

$${\mathbf{V}}_1=\sum _{i=0}^{\frac{n}{2}-1}{\left(1-\frac{1}{n}\right)}^{-1-2i+n}{\left(\frac{1}{n}\right)}^{1+2i}\left(\genfrac{}{}{0ex}{}{n}{1+2i}\right)=$$

$$=\frac{{\left(\frac{-1+n}{n}\right)}^n\left(-{\left(\frac{-2+n}{-1+n}\right)}^n+{\left(\frac{n}{-1+n}\right)}^n\right)}{2}$$

$${\mathbf{V}}_1=\sum _{i=0}^{\frac{n-1}{2}}{\left(1-\frac{1}{n}\right)}^{-1-2i+n}{\left(\frac{1}{n}\right)}^{1+2i}\left(\genfrac{}{}{0ex}{}{n}{1+2i}\right)=$$

$$=\frac{{\left(\frac{-1+n}{n}\right)}^n\left(-{\left(\frac{-2+n}{-1+n}\right)}^n+{\left(\frac{n}{-1+n}\right)}^n\right)}{2}$$

$$\underset{n\to \infty }{\lim }{\mathbf{V}}_1=\frac{1}{2}-\frac{1}{2e^2}=\mathrm{0.4323323...}$$

$$\frac{1}{2}-\frac{1}{2e^2}=\mathrm{0.4323323...}$$

$$\text{For all }i\in [k+1,n],{\overline{\mathbf{V}}}_i={\mathbf{V}}_i$$

Full text

AMS 2010 Mathematics Subject Classification 60G40, 62L15

1. Introduction

The Last-Success-Problem (LSP) is the problem of maximizing the probability of stopping on the last success in a finite sequence of Bernoulli trials. There are $n$ Bernoulli random variables which are observed sequentially. The problem is to find a stopping rule to maximize the probability of stopping at the last ”1”. This problem has been studied by Hill and Krengel [4] and Hsiau and Yang [5] for the case in which the random variables are independent and was simply and elegantly solved by T.F. Bruss in [1] with the following famous result.

Theorem 1**.**

(Odds-Theorem, T.F. Bruss 2000). Let $I_{1},I_{2},...,I_{n}$ be $n$ independent Bernoulli random variables with known $n$. We denote by ($i=1,...,n$) $p_{i}$ the parameter of $I_{i}$; i.e. ($p_{i}=P(I_{i}=1)$). Let $q_{i}=1-p_{i}$ and $r_{i}=p_{i}/q_{i}$. We define the index

[TABLE]

To maximize the probability of stopping on the last $"1"$ in the sequence, it is optimal to stop on the first $"1"$ that we encounter among the variables $I_{\mathbf{s}},I_{\mathbf{s}+1},...,I_{n}$.

The optimal win probability is given by

[TABLE]

We propose the following adversarial version of the problem in this paper. There are $n$ independent Bernoulli random variables $I_{i}$ with parameters $p_{i}$ that are observed sequentially. Two players, A and B, act in turns starting with A. After observing the value of $I_{k}$, if $I_{k}=1$, then the player whose turn it is may pass his turn to his opponent or use it and observe the variable $I_{k+1}$. When the last event is reached, if the result is success ($I_{n}=1$), the player whose turn it is wins, and loses otherwise. Specifically, if $I_{i}=0$ for all $i$, player A loses. This is reminiscent of the hot potato game in which the goal is not to be holding the hot potato at the end of the game, with the rule of being able to pass it on (if one so wishes) to one’s opponent when $I_{k}=1$.

Let us denote by $\mathbf{V}_{k}$ the probability of the player whose turn it is winning when we are about to observe the variable $I_{k}$. In particular, the probability of player A winning is $\mathbf{V}_{1}$; hence the probability of player B winning is $1-\mathbf{V}_{1}$. Likewise, on observing the last random variable, the player whose turn it is will win with probability $p_{k}$, i.e. $\mathbf{V}_{n}=p_{n}$.

The dynamic program to find the optimal strategy is straightforward. After observing the variable $I_{k}$, if $I_{k}=0$, which occurs with probability $1-p_{k}$, the player then irrevocably goes on to observe the variable $I_{k+1}$ without giving up his turn. If $I_{k}=1$, the optimal strategy of the player whose turn it is will consist in passing his turn to his opponent if $\mathbf{V}_{k+1}<\frac{1}{2}$ and in continuing with his turn if $\mathbf{V}_{k+1}\geq\frac{1}{2}$. We shall then have the following recurrence.

[TABLE]

2. Optimal strategy

We shall see that the optimal strategy is extremely simple and that it is also very easy to determine which of the two players has the greatest probability of winning. Another matter altogether is the exact calculation of this probability, which generally requires the computation of recurrence or calculations of the equivalent cost.

Proposition 1**.**

If for all $k\in[r,n]$, $p_{k}<\frac{1}{2}$, then for all $k\in[r,n]$ the following is fulfilled:

[TABLE]

Proof.

It is evident that $\mathbf{V}_{n}=p_{n}<\frac{1}{2}$. We proceed by backward induction. We assume that the proposition is true for all $i\in[k+1,n]$ and shall prove that it also holds for $i=k$. From the induction hypothesis, $\mathbf{V}_{k+1}<\frac{1}{2}$, therefore $1-\mathbf{V}_{k+1}>\frac{1}{2}>\mathbf{V}_{k+1}$, and hence

[TABLE]

On the other hand, considering that

[TABLE]

it turns out that

[TABLE]

∎

Proposition 2**.**

If $p_{r}>\frac{1}{2}$ and for all $k\in[r+1,n]$, $p_{k}<\frac{1}{2}$, then:

[TABLE]

Proof.

We will take into account that

[TABLE]

For Proposition 3, $\mathbf{V}_{r+1}<\frac{1}{2}$, so

[TABLE]

[TABLE]

Now, since $\mathbf{V}_{r}>\frac{1}{2}$, it is immediately followed by the dynamic program that

[TABLE]

∎

Proposition 3**.**

If $p_{r}=\frac{1}{2}$ and for all $k\in[r+1,n]$, $p_{k}<\frac{1}{2}$, then:

[TABLE]

Proof.

For Proposition 3, $\mathbf{V}_{r+1}<\frac{1}{2}$, so

[TABLE]

Now, since $\mathbf{V}_{r}=\frac{1}{2}$, it follows immediately from the dynamic program that

[TABLE]

∎

From the previous propositions the following result is followed without difficulty.

Proposition 4**.**

Let $\Omega_{r}:=\{k\in[r,n]:p_{k}\geq\frac{1}{2}\}$ and considering

[TABLE]

The optimal strategy for the player whose turn it is when observing the variable $I_{r}$ is not to give up his turn before stage $\mathfrak{u}_{r}$ and to do so when he may starting from $\mathfrak{u}_{r}$. In addition, the following is true.

$\bullet$* If $\Omega_{r}=\emptyset$, then $\mathbf{V}_{r}<\frac{1}{2}$.*

$\bullet$* If $\Omega_{r}\neq\emptyset$ and $p_{\mathfrak{u}_{r}}=\frac{1}{2}$, then $\mathbf{V}_{r}=\frac{1}{2}$.*

$\bullet$* If $\Omega_{r}\neq\emptyset$ and $p_{\mathfrak{u}_{r}}>\frac{1}{2}$, then $\mathbf{V}_{r}>\frac{1}{2}$.*

Let us denote by $\mathfrak{u}$ the optimal threshold of the first player in his first turn, $\mathfrak{u}:=\mathfrak{u}_{1}$ (the last Bernoulli event with parameter $\geq\frac{1}{2}$). The optimal strategy of the first player consists in continuing with his turn until reaching the $\mathfrak{u}$-th event and thereafter giving up his turn whenever possible. Obviously, player B will do the same in his optimal game because, when his turn comes, he will be in the same situation as player A. In short, we have the following result.

Theorem 2**.**

The optimal strategy for both players is to give up their turn when (and only when) there are no random variables left to observe whose parameter is greater than or equal to $\frac{1}{2}$.

In fact, when played optimally by both players, the game can be seen as a game of solitaire played by player A assuming his opponent uses the optimal strategy. Thus, the probability of player A winning, the optimal threshold being $\mathfrak{u}$, is the probability that the number of $1^{\prime}s$ starting from the $\mathfrak{u}$ resulting from the random variants is odd; that is to say:

[TABLE]

This allows establishing a (somewhat coarse) lower bound for the probability of player A winning. Bear in mind that the win probability in this game is greater than the probability of winning in the LSP.

Proposition 5**.**

If $\sum_{i=1}^{n}\frac{p_{i}}{1-p_{i}}\geq 1$, then

[TABLE]

Proof.

It suffices to keep in mind that the probability of winning in the LSP under these conditions is greater than $1/e$ (see [2]). ∎

3. All the random variables have the same parameter

In this section, we study the particular case that all the Bernoulli random variables have the same parameter.

Proposition 6**.**

If $p_{i}=p$ for all $i=1,...,n$, then the probability of player A winning is strictly increasing with $n$ always below its limit as $n$ tends to infinity

[TABLE]

Proof.

If $n$ is even, we have

[TABLE]

Similarly, if n is odd, we have

[TABLE]

∎

Proposition 7**.**

If we have $n$ Bernoulli random variables with $p_{i}=\frac{1}{n}$, the probability of player A winning is decreasing and is always greater that its limit as $n$ tends to infinity, namely

[TABLE]

Proof.

If $n=1$ then $\mathbf{V}_{1}=1$. If $n=2$ then $V_{2}=\frac{1}{2}$. If $n>3$, the optimal strategy for both players is to give up their turn whenever possible. Hence, player A will win if the number of $1^{\prime}s$ resulting from the random variables is odd.

If $n$ is even

[TABLE]

[TABLE]

Similarly, if $n$ is odd

[TABLE]

[TABLE]

[TABLE]

∎

Proposition 8**.**

If we have $n$ Bernoulli random variables with $p_{i}\geq\frac{1}{n}$, the probability of player A winning is greater than

[TABLE]

Proof.

If $p_{i}\geq\frac{1}{2}$ for some $i$, then $V_{1}\geq\frac{1}{2}$. Otherwise, think of the auxiliar game with all the parameters equal to $1/n$ in which the probability is greater than $\frac{1}{2}-\frac{1}{2\,e^{2}}$. Now, there is no more to considering successive modifications of this game, as in Lemma 1 (see below), with which the win probability increases, until reaching the game considered. ∎

Proposition 9**.**

If we have $n$ Bernoulli random variables, of which there are $m$ with $p_{i}\geq\frac{1}{m}$, the probability of player A winning is greater than

[TABLE]

Proof.

It can easily be seen that if $p_{i}<\frac{1}{2}$ for all $i$, then the probability of player A winning is greater than the probability that he would have in the game resulting from excluding some random variable. Consequently, it suffices to observe that the value, $\frac{1}{2}-\frac{1}{2\,e^{2}}$, is exceeded in the auxiliary game resulting from excluding some random variables. In fact, if we have $m$ variables with $p_{i}<1/m$ for all of these variables, considering the game in which the other variables are excluded, then we are able to use the previous proposition. ∎

Lemma 1**.**

Let us consider the game with parameters $p_{i}<\frac{1}{2}$ and denote by $\overline{\mathbf{V}}_{i}$ the probability of a player winning when it is his turn after observing the variable $I_{i}$ in the resulting auxiliary game when changing $p_{k}$ to $\overline{p}_{k}>p_{k}$. Hence,

[TABLE]

[TABLE]

In other words, if we increase the value of the parameter of one of the Bernoulli random variables in a game, then the player’s probability of winning on his turn increases.

Proof.

Let us recall that $\mathbf{V}_{i}$ and $\overline{\mathbf{V}}_{i}$ respectively denote the player’s probability of winning on his turn at stage $i$ in the original game and in the auxiliary game.

$\bullet$ For all $i>k$, it is evident that $\mathbf{V}_{i}=\overline{\mathbf{V}}_{i}$ as we are in a subsequent stage to the modified variable and the process “has no memory” and therefore does not affect.

$\bullet$ For all $i\leq k$, we will proceed by induction backwards. Let us first see that it is true for $i=k$.

[TABLE]

[TABLE]

[TABLE]

We now assume that the proposal is fulfilled for $i+1$ and shall prove that it is fulfilled for $i$

[TABLE]

[TABLE]

[TABLE]

∎

3.1. A variant: If there have been no $1^{\prime}s$, the game is

repeated

We have seen that the game is advantageous for player A if and only if some parameter is greater than $\frac{1}{2}$. The reason that the game is disadvantageous for player A is related to the fact that he can lose because the results of all the random variables are [math]. In fact, if any of the variables is worth $1$, then the probability of the player winning by giving up his turn is greater than $\frac{1}{2}$. The following result shows that, if the rule of repeating the game is introduced and if there have been no $1^{\prime}s$, then the game is very advantageous for player A.

Proposition 10**.**

If $p_{i}=1/n$ for all $i$ and considering the rule that the game is repeated in the case of $I_{i}=0$ for all $i$, the probability of player A winning is

[TABLE]

Besides, we have that $\mathcal{V}(n)$ is increasing and

[TABLE]

Proof.

Obviously, the optimal strategy with this rule is the same as in the game in its original version. The difference lies only in the probability of winning, which is conditioned by $\sum_{i=1}^{n}I_{i}>0$. Thus, bearing in mind that

[TABLE]

we have that

[TABLE]

Moreover, $\mathcal{V}(n)$ is increasing and its limit is $\frac{1+e}{2\,e}$. ∎

4. Random parameters for the Bernoulli variables

We finally determine the probability of player A winning (mean probability) when the parameters are the results of uniform random variables, $\mathbf{U}[0,1]$. That is to say, before holding the competition, the parameters of the Bernoulli variables are drawn via random trials of a uniform random variable $\mathbf{U}[0,1]$ and these parameters are revealed to the players.

Lemma 2**.**

If the last $k$ variables $\{I_{i}\}_{i=n-k+1}^{n}$ have parameters $p_{i}$, which are the results of the uniform random variables $\mathbf{U}[0,\frac{1}{2}]$, then the win probability of the player whose turn it is at stage $n-k+1$ is

[TABLE]

In particular, if $k=n$

[TABLE]

Proof.

We denote by $X_{i}$ the win probability of player whose turn it is at stage $n-i+1$. Bearing in mind that $t=p_{n-k+1}$ is the result of a uniform random variable, $\mathbf{U}[0,\frac{1}{2}]$

[TABLE]

[TABLE]

and solving with $X_{0}=0$ gives

[TABLE]

∎

Lemma 3**.**

If we have $k-1$ end variables with parameters $p_{i}$ that are the result of uniform random variables, $\mathbf{U}[0,\frac{1}{2}]$, and $p_{k}$ is the result of a uniform random variable, $\mathbf{U}[\frac{1}{2},1]$, then the probability of player A winning is

[TABLE]

Proof.

We denote by $J_{k}$ the win probability of the player whose turn it is after $n-k+1$. Reasoning similar to above

[TABLE]

[TABLE]

[TABLE]

∎

Lemma 4**.**

If all the parameters $p_{i}$ are the result of uniform random variables, $\mathbf{U}[0,\frac{1}{2}]$, then player A’s probability of winning is

[TABLE]

Proposition 11**.**

If we have a game with $n$ variables whose parameters are the result of the $n$ uniform random variables, $\mathbf{U}[0,1]$, player A’s probability of winning is

[TABLE]

Proof.

The probability that the last parameter greater than or equal to $\frac{1}{2}$ will be the $k$-th is $2^{-n+k-1}$ and the probability that all the parameters are less than $\frac{1}{2}$ will be $2^{-n}$.

[TABLE]

∎

5. Conclusions and future challenges

The proposed adversarial version of the Last-Success-Problem has a very simple optimal game strategy that does not require any calculation. It only requires identifying the last variable whose parameter is greater than $\frac{1}{2}$ and, as from that point on, always giving up one’s turn to one’s opponent. It seems interesting to pose the problem with non-independent random variables. The Last-Success-Problem with dependent Bernoulli random variables was addressed by Tamaki in [8], who considered that $I_{1},...,I_{n}$ constitute a Markov chain with transition probabilities

[TABLE]

[TABLE]

and established an optimal stopping rule with a Markov version of the odds-theorem.

We predict that the adversarial version with dependent variables will also be simple and the optimal strategy will most likely consist in adopting, at each $k$-th stage, the optimal strategy while assuming that the remaining variables are independent $\widehat{I}_{k+1},...,\widehat{I}_{n}$ with parameters (computable by recurrence)

[TABLE]

In short, we conjecture the following.

Conjecture 1**.**

Let $I_{1},I_{2},...,I_{n}$ be $n$ dependent Bernoulli random variables. Let $p_{i,k}:=P(I_{i}=1|I_{k}=1)$. Then, the optimal strategy for the player whose turn it is after observing the variable $I_{k}=1$ is to give up his turn to his opponent if and only if $p_{i,k}<\frac{1}{2}$ for all $i>k$.

It may also be interesting to pose the game with more than 2 players, in which case different types of payment could be considered. For any version, it is normal to consider the loser to be the player whose turn it is after the last Bernoulli trial, but several types of payment may be considered for the other players. If we consider that the players who do not lose each receive the same payment, we have the simplest version. In this respect, we conclude by posing the challenge to determine the limit with $m$ players, when $n$ tends to infinity, of the loss probability of each player, considering $n$ independent Bernoulli random variables with parameters $1/n$. In fact, for 3 players, it is no longer a trivial problem, as only the limit for the probability of the first player losing is exactly calculable in a relatively straightforward way. Using the Mathematica symbolic calculation package, we obtained the following limit for the probability of the first player losing:

[TABLE]

However, it is no longer viable to find the exact limit of the probability of the other two players losing via this path. Computing for large values of $n$ allows an approximation, but only that. Specifically, we have that:

[TABLE]

In all the above calculations, we have assumed that the optimal strategy for both players is to give up their turn whenever possible. Of course, this will undoubtedly be true in this case. In general, however, there will be an optimal strategy that does not always consist in passing one’s turn to the following player.