Accelerating Self-Play Learning in Go

TL;DR

This paper presents KataGo, a self-play learning algorithm for Go that significantly accelerates training, reducing computational requirements by 50 times compared to previous methods like AlphaZero and ELF OpenGo, through general and domain-specific improvements.

Contribution

The authors introduce a set of improvements to the AlphaZero framework that greatly speed up self-play learning in Go, making it feasible with much less computational resources.

Findings

KataGo achieves comparable performance to ELF OpenGo after only 19 days on fewer than 30 GPUs.

The speedup involves non-domain-specific improvements transferable to other problems.

Domain-specific techniques further close the gap between general methods and state-of-the-art performance.

Abstract

By introducing several improvements to the AlphaZero process and architecture, we greatly accelerate self-play learning in Go, achieving a 50x reduction in computation over comparable methods. Like AlphaZero and replications such as ELF OpenGo and Leela Zero, our bot KataGo only learns from neural-net-guided Monte Carlo tree search self-play. But whereas AlphaZero required thousands of TPUs over several days and ELF required thousands of GPUs over two weeks, KataGo surpasses ELF's final model after only 19 days on fewer than 30 GPUs. Much of the speedup involves non-domain-specific improvements that might directly transfer to other problems. Further gains from domain-specific techniques reveal the remaining efficiency gap between the best methods and purely general methods such as AlphaZero. Our work is a step towards making learning in state spaces as large as Go possible without large-scale computational resources.