ColosseumRL: A Framework for Multiagent Reinforcement Learning in $N$-Player Games

TL;DR

This paper introduces ColosseumRL, a new framework designed for multiagent reinforcement learning in n-player general sum games, aiming to facilitate research beyond the traditional two-player zero-sum setting.

Contribution

The paper presents a novel framework for studying reinforcement learning in complex n-player games, addressing the gap in solution concepts for such environments.

Findings

Framework enables analysis of multiagent behaviors in n-player games

Supports research towards meaningful solution concepts in general sum games

Provides a platform for advancing multiagent reinforcement learning research

Abstract

Much of recent success in multiagent reinforcement learning has been in two-player zero-sum games. In these games, algorithms such as fictitious self-play and minimax tree search can converge to an approximate Nash equilibrium. While playing a Nash equilibrium strategy in a two-player zero-sum game is optimal, in an $n$-player general sum game, it becomes a much less informative solution concept. Despite the lack of a satisfying solution concept, $n$-player games form the vast majority of real-world multiagent situations. In this paper we present a new framework for research in reinforcement learning in $n$-player games. We hope that by analyzing behavior learned by agents in these environments the community can better understand this important research area and move toward meaningful solution concepts and research directions. The implementation and additional information about this framework can be found at https://colosseumrl.igb.uci.edu/.