Word Play for Playing Othello (Reverses)

TL;DR

This paper explores using GPT-2 and GPT-3 language models to generate and analyze Othello game strategies, achieving high game completion rates and expanding available game archives for research.

Contribution

It demonstrates that large language models can emulate championship-level Othello strategies and significantly increase the size of available game archives for research.

Findings

GPT-2 generates 13-71% complete games

GPT-3 reaches 41% game completion

Models double the previous game archive record

Abstract

Language models like OpenAI's Generative Pre-Trained Transformers (GPT-2/3) capture the long-term correlations needed to generate text in a variety of domains (such as language translators) and recently in gameplay (chess, Go, and checkers). The present research applies both the larger (GPT-3) and smaller (GPT-2) language models to explore the complex strategies for the game of Othello (or Reverses). Given the game rules for rapid reversals of fortune, the language model not only represents a candidate predictor of the next move based on previous game moves but also avoids sparse rewards in gameplay. The language model automatically captures or emulates championship-level strategies. The fine-tuned GPT-2 model generates Othello games ranging from 13-71% completion, while the larger GPT-3 model reaches 41% of a complete game. Like previous work with chess and Go, these language models offer a novel way to generate plausible game archives, particularly for comparing opening moves across a larger sample than humanly possible to explore. A primary contribution of these models magnifies (by two-fold) the previous record for player archives (120,000 human games over 45 years from 1977-2022), thus supplying the research community with more diverse and original strategies for sampling with other reinforcement learning techniques.