Decentralized Stochastic Control with Partial History Sharing: A Common Information Approach

TL;DR

This paper introduces a unified framework for decentralized stochastic control with partial history sharing, reformulating the problem as a centralized POMDP to derive optimal strategies and structural results.

Contribution

It presents a common information approach that generalizes existing models, providing a simpler dynamic programming solution and novel structural insights for decentralized control.

Findings

Unified framework for partial history sharing models

Structural results for optimal strategies derived

Simpler dynamic programming approach obtained

Abstract

A general model of decentralized stochastic control called partial history sharing information structure is presented. In this model, at each step the controllers share part of their observation and control history with each other. This general model subsumes several existing models of information sharing as special cases. Based on the information commonly known to all the controllers, the decentralized problem is reformulated as an equivalent centralized problem from the perspective of a coordinator. The coordinator knows the common information and select prescriptions that map each controller's local information to its control actions. The optimal control problem at the coordinator is shown to be a partially observable Markov decision process (POMDP) which is solved using techniques from Markov decision theory. This approach provides (a) structural results for optimal strategies, and (b) a dynamic program for obtaining optimal strategies for all controllers in the original decentralized problem. Thus, this approach unifies the various ad-hoc approaches taken in the literature. In addition, the structural results on optimal control strategies obtained by the proposed approach cannot be obtained by the existing generic approach (the person-by-person approach) for obtaining structural results in decentralized problems; and the dynamic program obtained by the proposed approach is simpler than that obtained by the existing generic approach (the designer's approach) for obtaining dynamic programs in decentralized problems.