Surprising strategies obtained by stochastic optimization in partially observable games

TL;DR

This paper compares five algorithms for optimizing strategies in two-player zero-sum games with incomplete information, revealing unexpected strategies and highlighting the effectiveness of a simple seed method across various games.

Contribution

It introduces and evaluates multiple strategy optimization algorithms, including coevolution and seed methods, demonstrating their performance and uncovering surprising strategies in complex games.

Findings

Coevolution methods performed well but lacked stability.

The seed method was simple yet effective across different games.

Unexpected strategies emerged, such as in Batawaf and War.

Abstract

This paper studies the optimization of strategies in the context of possibly randomized two players zero-sum games with incomplete information. We compare 5 algorithms for tuning the parameters of strategies over a benchmark of 12 games. A first evolutionary approach consists in designing a highly randomized opponent (called naive opponent) and optimizing the parametric strategy against it; a second one is optimizing iteratively the strategy, i.e. constructing a sequence of strategies starting from the naive one. 2 versions of coevolutions, real and approximate, are also tested as well as a seed method. The coevolution methods were performing well, but results were not stable from one game to another. In spite of its simplicity, the seed method, which can be seen as an extremal version of coevolution, works even when nothing else works. Incidentally, these methods brought out some unexpected strategies for some games, such as Batawaf or the game of War, which seem, at first view, purely random games without any structured actions possible for the players or Guess Who, where a dichotomy between the characters seems to be the most reasonable strategy. All source codes of games are written in Matlab/Octave and are freely available for download.