Learning to Play Stochastic Two-player Perfect-Information Games without Knowledge

TL;DR

This paper extends the Descent framework to stochastic two-player perfect-information games, proposing two methods and demonstrating superior performance of the generalized approach in the game EinStein wurfelt nicht! compared to existing algorithms.

Contribution

The paper introduces a novel generalization of the Descent algorithm to stochastic games and compares it with an approximation method, advancing learning without prior knowledge.

Findings

The generalized Descent algorithm outperforms Expectiminimax and Polygames in the tested game.

The deterministic approximation method yields promising results, indicating potential in specific contexts.

The approach enables learning and planning in stochastic games without prior domain knowledge.

Abstract

In this paper, we extend the Descent framework, which enables learning and planning in the context of two-player games with perfect information, to the framework of stochastic games. We propose two ways of doing this, the first way generalizes the search algorithm, i.e. Descent, to stochastic games and the second way approximates stochastic games by deterministic games. We then evaluate them on the game EinStein wurfelt nicht! against state-of-the-art algorithms: Expectiminimax and Polygames (i.e. the Alpha Zero algorithm). It is our generalization of Descent which obtains the best results. The approximation by deterministic games nevertheless obtains good results, presaging that it could give better results in particular contexts.