Optimality of Symmetric Independent Policies under Decentralized Mean-Field Information Sharing for Stochastic Teams and Equivalence with McKean-Vlasov Control of a Representative Agent

TL;DR

This paper proves that symmetric, independent policies are optimal for infinite-agent mean-field teams and shows their near-optimality for large finite teams, establishing a link with McKean-Vlasov control.

Contribution

It demonstrates the existence and optimality of symmetric, decentralized policies in mean-field stochastic teams and connects finite-agent optimal policies to infinite-agent limits.

Findings

Optimal symmetric policies exist for finite teams.

Finite optimal policies converge to infinite-team optimal policies.

Symmetric policies are approximately optimal for large finite teams.

Abstract

We study a class of stochastic exchangeable teams with a finite number of decision makers (DMs) as well as their mean-field limits with infinitely many DMs. In the finite population regime, we study exchangeable teams under the centralized information structure. The paper makes the following main contributions: i) For finite population exchangeable teams, we establish the existence of an optimal policy that is exchangeable (permutation invariant) and Markovian; ii) As our main result in the paper, we show that a sequence of exchangeable optimal policies for finite population settings (which satisfies a measure valued MDP formulation due to B{\"a}uerle) converges to a decentralized symmetric (identical) and conditionally independent (given the mean-field) policy for the infinite population problem, which is then globally optimal under both the centralized information structure as well as the mean-field sharing information structure. (iii) This result establishes existence of a symmetric, independent, decentralized optimal randomized policy for the infinite population problem and proves the optimality of the limiting measure-valued MDP for the representative DM. Our paper thus establishes the relation between the controlled McKean-Vlasov dynamics and the optimal infinite population decentralized stochastic control problem (without an apriori restriction of symmetry in policies of individual agents), for the first time, to our knowledge (beyond several special cases). We also establish near optimality of a numerical method for solving this problem. iv) Finally, we show that symmetric, independent, decentralized optimal randomized policies are approximately optimal for the corresponding finite-population team with a large number of DMs under the centralized information structure.