A Variational Inequality Approach to Independent Learning in Static Mean-Field Games

TL;DR

This paper models large-scale multi-agent games as variational inequalities, proposing independent learning algorithms with convergence guarantees, validated through simulations and real-world applications in traffic and network management.

Contribution

It introduces a variational inequality framework for static mean-field games and develops independent learning algorithms with finite-sample guarantees.

Findings

Algorithms converge to approximate Nash equilibria.

Finite sample complexity guarantees are established.

Validated through simulations and real-world case studies.

Abstract

Competitive games involving thousands or even millions of players are prevalent in real-world contexts, such as transportation, communications, and computer networks. However, learning in these large-scale multi-agent environments presents a grand challenge, often referred to as the "curse of many agents". In this paper, we formalize and analyze the Static Mean-Field Game (SMFG) under both full and bandit feedback, offering a generic framework for modeling large population interactions while enabling independent learning. We first establish close connections between SMFG and variational inequality (VI), showing that SMFG can be framed as a VI problem in the infinite agent limit. Building on the VI perspective, we propose independent learning and exploration algorithms that efficiently converge to approximate Nash equilibria, when dealing with a finite number of agents. Theoretically, we provide explicit finite sample complexity guarantees for independent learning across various feedback models in repeated play scenarios, assuming (strongly-)monotone payoffs. Numerically, we validate our results through both simulations and real-world applications in city traffic and network access management.