MEMOA: Massive Mixtures of Online Agents via Mean-Field Decentralized Nash Equilibria

TL;DR

This paper introduces MEMOA, a decentralized approach for large-scale AI agent training that optimally balances individual and collective performance using mean-field theory, with proven convergence to Nash equilibrium.

Contribution

It derives a closed-form optimal decentralized policy for large populations, converging to Nash equilibrium, and improves mean predictions via online weighting.

Findings

Decentralized policy outperforms greedy baselines in experiments.

Policy converges asymptotically to Nash equilibrium in large populations.

Numerical results verify theoretical guarantees.

Abstract

In the modern age of large-scale AI, federated learning has become an increasingly important tool for training large populations of AI agents; however, its computational and communication costs can rapidly fail to scale with the number of agents. This is precisely where decentralized agentic strategies shine: each agent acts autonomously, using only its own state together with a minimal summary of the ensemble, namely the mean-field. We derive the unique optimal decentralized policy in closed form. Optimality is characterized through a worst-client/minimax criterion: minimizing the under-performer regret, namely the maximal online cost incurred by the weakest agent in the ensemble. We further prove that the resulting decentralized policy asymptotically converges, in the large-population limit, to the Nash-optimal centralized policy, whose direct computation is not scalable. We use an online weighting mechanism to optimize the server-computed mixture of client predictions, thereby improving the mean prediction in addition to the previously optimized weakest-client prediction. Numerical experiments verify our theoretical guarantees and demonstrate that our decentralized policy typically outperforms natural greedy decentralized baselines.