Optimal strategies in the all-heads coin game

TL;DR

This paper analyzes a sequential coin-flipping game, determining optimal strategies and winning probabilities based on the number of coins and bias, with explicit formulas and perturbation analysis near p=1/2.

Contribution

It provides a detailed analysis of optimal strategies and explicit formulas for winning probabilities in the all-heads coin game, including perturbation results near p=1/2.

Findings

For p=1/2, all strategies have a 50% winning probability.

For p>1/2, the 'set aside one head' strategy is optimal.

Near p=1/2, the winning probability exhibits a linear perturbation with a specific limit.

Abstract

We study a sequential coin-flipping game in which a player starts with~$n$ coins, each landing heads independently with probability~$p$. In each round the player flips all remaining coins and must set aside at least one coin showing heads; if no coin shows heads, the player loses. The player wins if and when all coins have been set aside. This is a stochastic shortest-path Markov decision process whose Bellman equation involves a nonlinear suffix-maximum operator. We analyse two natural strategies -- \One{} (set aside exactly one head) and \All{} (set aside every head) -- and determine the optimal winning probability~$w_{n,p}$ as a function of~$n$ and~$p$. For $p=\tfrac12$ every strategy achieves $w_{n,1/2}=\tfrac12$. For $p>\tfrac12$ the strategy~\One{} is optimal and $n\mapsto w_{n,p}$ is strictly increasing; as a consequence the limit $W(p):=\lim_n w_{n,p}$ admits an explicit series representation and satisfies $p\le W(p)<1$. For $p<\tfrac12$ near~$\tfrac12$ we carry out a first-order perturbation expansion in $\delta:=\tfrac12-p$ and prove that the deficit is $\tfrac12-w_{n,\tfrac 12 - \delta} \approx \delta c_n$ in first order in $\delta$, where $c_n$ satisfies a linear recursion for $n\ge 7$, with a limit $L=\lim c_n\approx 1.7035$. As a consequence the optimal value sequence has, to first order in~$\delta$, a strict local minimum at $n=5$ and no local maximum -- local maxima are a non-perturbative phenomenon.