A finite exact algorithm to solve a dice game

TL;DR

This paper introduces a finite, exact algorithm for determining the optimal strategy and value in a solitaire dice game, leveraging Markov Control Processes and a novel critical threshold concept.

Contribution

It presents a new finite and exact algorithm for solving a dice game using Markov Control Processes and a critical threshold approach.

Findings

The algorithm precisely computes the game's value and optimal strategy.

The set of non-stopping states is finite once the critical threshold is identified.

The approach is based on a backward induction method using continuous pasting conditions.

Abstract

We provide an algorithm to find the value and an optimal strategy of the solitaire variant of the Ten Thousand dice game in the framework of Markov Control Processes. Once an optimal critical threshold is found, the set of non-stopping states of the game becomes finite, and the solution is found by a backwards algorithm that gives the values for each one of these states of the game. The algorithm is finite and exact.The idea to find the critical threshold comes from the continuous pasting condition used in optimal stopping problems for continuous-time processes with jumps.