OLIVAW: Mastering Othello without Human Knowledge, nor a Fortune

TL;DR

OLIVAW demonstrates that applying AlphaGo Zero's reinforcement learning paradigm to Othello enables the creation of a highly competent AI using only affordable hardware and free cloud resources, surpassing traditional engines and human players.

Contribution

This work adapts AlphaGo Zero's approach to Othello, introducing specific improvements to accelerate learning and achieve high-level play without human knowledge or extensive resources.

Findings

OLIVAW outperforms the strongest open-source Othello engine Edax.

OLIVAW defeats top human players in matches.

The approach requires only commodity hardware and free cloud services.

Abstract

We introduce OLIVAW, an AI Othello player adopting the design principles of the famous AlphaGo programs. The main motivation behind OLIVAW was to attain exceptional competence in a non-trivial board game at a tiny fraction of the cost of its illustrious predecessors. In this paper, we show how the AlphaGo Zero's paradigm can be successfully applied to the popular game of Othello using only commodity hardware and free cloud services. While being simpler than Chess or Go, Othello maintains a considerable search space and difficulty in evaluating board positions. To achieve this result, OLIVAW implements some improvements inspired by recent works to accelerate the standard AlphaGo Zero learning process. The main modification implies doubling the positions collected per game during the training phase, by including also positions not played but largely explored by the agent. We tested the strength of OLIVAW in three different ways: by pitting it against Edax, the strongest open-source Othello engine, by playing anonymous games on the web platform OthelloQuest, and finally in two in-person matches against top-notch human players: a national champion and a former world champion.