Mean-Field Control on Sparse Graphs: From Local Limits to GNNs via Neighborhood Distributions

TL;DR

This paper develops a theoretical framework for mean-field control on large sparse graphs, enabling scalable reinforcement learning and justifying the use of GNNs for multi-agent systems with local interactions.

Contribution

It introduces a new probabilistic state representation over neighborhood distributions and proves horizon-dependent locality, facilitating scalable control on sparse networks.

Findings

Horizon-dependent locality theorem for finite-horizon problems

A new Dynamic Programming Principle on neighborhood distributions

Experimental validation of GNNs for actor-critic algorithms in sparse graph control

Abstract

Mean-field control (MFC) offers a scalable solution to the curse of dimensionality in multi-agent systems but traditionally hinges on the restrictive assumption of exchangeability via dense, all-to-all interactions. In this work, we bridge the gap to real-world network structures by proposing a rigorous framework for MFC on large sparse graphs. We redefine the system state as a probability measure over decorated rooted neighborhoods, effectively capturing local heterogeneity. Our central contribution is a theoretical foundation for scalable reinforcement learning in this setting. We prove horizon-dependent locality: for finite-horizon problems, an agent's optimal policy at time t depends strictly on its (T-t)-hop neighborhood. This result renders the infinite-dimensional control problem tractable and underpins a novel Dynamic Programming Principle (DPP) on the lifted space of neighborhood distributions. Furthermore, we formally and experimentally justify the use of Graph Neural Networks (GNNs) for actor-critic algorithms in this context. Our framework naturally recovers classical MFC as a degenerate case while enabling efficient, theoretically grounded control on complex sparse topologies.