# $\alpha$-Rank: Multi-Agent Evaluation by Evolution

## TL;DR

$ ext{ extalpha}$-Rank is a scalable evolutionary dynamics method for evaluating and ranking agents in large multi-agent systems, providing insights into their strategic strengths and long-term behaviors based on a novel game-theoretic solution concept.

## Contribution

The paper introduces $ ext{ extalpha}$-Rank, a new evolutionary evaluation framework based on Markov-Conley chains, scalable to large multi-agent interactions and guaranteed to converge to a dynamical solution concept.

## Key findings

- $ ext{ extalpha}$-Rank effectively ranks agents in complex multi-agent environments.
- The method scales polynomially with the number of strategies, unlike Nash equilibrium.
- Empirical validation on AlphaGo, AlphaZero, MuJoCo Soccer, and Poker demonstrates its practical utility.

## Abstract

We introduce $\alpha$-Rank, a principled evolutionary dynamics methodology for the evaluation and ranking of agents in large-scale multi-agent interactions, grounded in a novel dynamical game-theoretic solution concept called Markov-Conley chains (MCCs). The approach leverages continuous- and discrete-time evolutionary dynamical systems applied to empirical games, and scales tractably in the number of agents, the type of interactions, and the type of empirical games (symmetric and asymmetric). Current models are fundamentally limited in one or more of these dimensions and are not guaranteed to converge to the desired game-theoretic solution concept (typically the Nash equilibrium). $\alpha$-Rank provides a ranking over the set of agents under evaluation and provides insights into their strengths, weaknesses, and long-term dynamics. This is a consequence of the links we establish to the MCC solution concept when the underlying evolutionary model's ranking-intensity parameter, $\alpha$, is chosen to be large, which exactly forms the basis of $\alpha$-Rank. In contrast to the Nash equilibrium, which is a static concept based on fixed points, MCCs are a dynamical solution concept based on the Markov chain formalism, Conley's Fundamental Theorem of Dynamical Systems, and the core ingredients of dynamical systems: fixed points, recurrent sets, periodic orbits, and limit cycles. $\alpha$-Rank runs in polynomial time with respect to the total number of pure strategy profiles, whereas computing a Nash equilibrium for a general-sum game is known to be intractable. We introduce proofs that not only provide a unifying perspective of existing continuous- and discrete-time evolutionary evaluation models, but also reveal the formal underpinnings of the $\alpha$-Rank methodology. We empirically validate the method in several domains including AlphaGo, AlphaZero, MuJoCo Soccer, and Poker.