Bayesian Optimization in AlphaGo

TL;DR

This paper discusses how Bayesian optimization was used to tune hyper-parameters in AlphaGo, significantly improving its playing strength and win-rate during development and competitive matches.

Contribution

It presents a case study of applying Bayesian optimization to tune AlphaGo's hyper-parameters, demonstrating substantial performance improvements.

Findings

Win-rate increased from 50% to 66.5% after tuning.

Bayesian optimization contributed significantly to AlphaGo's strength.

Multiple tuning cycles amplified overall performance gains.

Abstract

During the development of AlphaGo, its many hyper-parameters were tuned with Bayesian optimization multiple times. This automatic tuning process resulted in substantial improvements in playing strength. For example, prior to the match with Lee Sedol, we tuned the latest AlphaGo agent and this improved its win-rate from 50% to 66.5% in self-play games. This tuned version was deployed in the final match. Of course, since we tuned AlphaGo many times during its development cycle, the compounded contribution was even higher than this percentage. It is our hope that this brief case study will be of interest to Go fans, and also provide Bayesian optimization practitioners with some insights and inspiration.