FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning

TL;DR

FightLadder introduces a new benchmark platform for competitive multi-agent reinforcement learning using a real-time fighting game, facilitating evaluation of algorithms in challenging, visually rich environments.

Contribution

The paper presents FightLadder, an open-source fighting game platform with integrated algorithms and metrics, filling a gap in benchmarks for competitive MARL research.

Findings

A general agent defeats 12 characters in single-player mode.

Training a non-exploitable agent in two-player mode remains challenging.

FightLadder enables evaluation of agent performance and exploitability.

Abstract

Recent advances in reinforcement learning (RL) heavily rely on a variety of well-designed benchmarks, which provide environmental platforms and consistent criteria to evaluate existing and novel algorithms. Specifically, in multi-agent RL (MARL), a plethora of benchmarks based on cooperative games have spurred the development of algorithms that improve the scalability of cooperative multi-agent systems. However, for the competitive setting, a lightweight and open-sourced benchmark with challenging gaming dynamics and visual inputs has not yet been established. In this work, we present FightLadder, a real-time fighting game platform, to empower competitive MARL research. Along with the platform, we provide implementations of state-of-the-art MARL algorithms for competitive games, as well as a set of evaluation metrics to characterize the performance and exploitability of agents. We demonstrate the feasibility of this platform by training a general agent that consistently defeats 12 built-in characters in single-player mode, and expose the difficulty of training a non-exploitable agent without human knowledge and demonstrations in two-player mode. FightLadder provides meticulously designed environments to address critical challenges in competitive MARL research, aiming to catalyze a new era of discovery and advancement in the field. Videos and code at https://sites.google.com/view/fightladder/home.