Decentralized Exchangeable Stochastic Dynamic Teams in Continuous-time, their Mean-Field Limits and Optimality of Symmetric Policies

TL;DR

This paper investigates stochastic exchangeable teams with finite and infinite decision makers, establishing the existence and convergence of optimal policies, and demonstrating their approximate optimality in large finite populations.

Contribution

It proves the existence of symmetric, decentralized optimal policies for infinite populations and their convergence from finite population solutions, linking McKean-Vlasov dynamics to stochastic control.

Findings

Existence of exchangeable, Markovian optimal policies in finite populations.

Convergence of finite population policies to a decentralized infinite population policy.

Approximate optimality of decentralized policies in large finite populations.

Abstract

We study a class of stochastic exchangeable teams comprising a finite number of decision makers (DMs) as well as their mean-field limits involving infinite numbers of DMs. In the finite population regime, we study exchangeable teams under the centralized information structure. For the infinite population setting, we study exchangeable teams under the decentralized mean-field information sharing. The paper makes the following main contributions: i) For finite population exchangeable teams, we establish the existence of a randomized optimal policy that is exchangeable (permutation invariant) and Markovian; ii) As our main result in the paper, we show that a sequence of exchangeable optimal policies for finite population settings converges to a conditionally symmetric (identical), independent, and decentralized randomized policy for the infinite population problem, which is globally optimal for the infinite population problem. This result establishes the existence of a symmetric, independent, decentralized optimal randomized policy for the infinite population problem. Additionally, this proves the optimality of the limiting measure-valued MDP for the representative DM; iii) Finally, we show that symmetric, independent, decentralized optimal randomized policies are approximately optimal for the corresponding finite-population team with a large number of DMs under the centralized information structure. Our paper thus establishes the relation between the controlled McKean-Vlasov dynamics and the optimal infinite population decentralized stochastic control problem (without an apriori restriction of symmetry in policies of individual agents), for the first time, to our knowledge.