Multiagent Learning in Large Anonymous Games

TL;DR

This paper investigates scalable learning algorithms in large anonymous games, demonstrating that simple stage learning converges efficiently to Nash equilibria under certain conditions, with insights on how agent count and information sharing affect convergence.

Contribution

It introduces conditions under which stage learning converges in large anonymous games and identifies features that enhance convergence efficiency.

Findings

More agents can improve convergence in many settings.

Providing statistical information reduces observations needed.

Stage learning converges efficiently when best-reply dynamics do.

Abstract

In large systems, it is important for agents to learn to act effectively, but sophisticated multi-agent learning algorithms generally do not scale. An alternative approach is to find restricted classes of games where simple, efficient algorithms converge. It is shown that stage learning efficiently converges to Nash equilibria in large anonymous games if best-reply dynamics converge. Two features are identified that improve convergence. First, rather than making learning more difficult, more agents are actually beneficial in many settings. Second, providing agents with statistical information about the behavior of others can significantly reduce the number of observations needed.