Final Adaptation Reinforcement Learning for N-Player Games

TL;DR

This paper introduces Final Adaptation RL (FARL), a novel algorithmic enhancement for n-player reinforcement learning that improves strategy quality across various board games by incorporating player-centered reward propagation.

Contribution

The paper presents FARL, a new element integrated into existing RL algorithms, enabling effective learning in multi-player games with a player-centered reward propagation approach.

Findings

FARL significantly improves learning outcomes in multiple n-player games.

Algorithms with FARL achieve near-perfect strategies in tested board games.

FARL is essential for success in the proposed player-centered RL framework.

Abstract

This paper covers n-tuple-based reinforcement learning (RL) algorithms for games. We present new algorithms for TD-, SARSA- and Q-learning which work seamlessly on various games with arbitrary number of players. This is achieved by taking a player-centered view where each player propagates his/her rewards back to previous rounds. We add a new element called Final Adaptation RL (FARL) to all these algorithms. Our main contribution is that FARL is a vitally important ingredient to achieve success with the player-centered view in various games. We report results on seven board games with 1, 2 and 3 players, including Othello, ConnectFour and Hex. In most cases it is found that FARL is important to learn a near-perfect playing strategy. All algorithms are available in the GBG framework on GitHub.