Choosing Well Your Opponents: How to Guide the Synthesis of Programmatic Strategies

TL;DR

This paper presents the Local Learner (2L) algorithm that enhances the synthesis of programmatic strategies in two-player zero-sum games by actively selecting reference strategies, outperforming existing methods and human-designed strategies.

Contribution

The paper introduces 2L, a novel algorithm that improves search guidance in strategy synthesis by actively selecting reference strategies, outperforming traditional algorithms like IBR, FP, and DO.

Findings

2L provides stronger search signals than IBR, FP, and DO.

2L outperforms human-designed strategies in MicroRTS tournaments.

Empirical results demonstrate 2L's effectiveness across multiple games.

Abstract

This paper introduces Local Learner (2L), an algorithm for providing a set of reference strategies to guide the search for programmatic strategies in two-player zero-sum games. Previous learning algorithms, such as Iterated Best Response (IBR), Fictitious Play (FP), and Double-Oracle (DO), can be computationally expensive or miss important information for guiding search algorithms. 2L actively selects a set of reference strategies to improve the search signal. We empirically demonstrate the advantages of our approach while guiding a local search algorithm for synthesizing strategies in three games, including MicroRTS, a challenging real-time strategy game. Results show that 2L learns reference strategies that provide a stronger search signal than IBR, FP, and DO. We also simulate a tournament of MicroRTS, where a synthesizer using 2L outperformed the winners of the two latest MicroRTS competitions, which were programmatic strategies written by human programmers.