Decentralized Control of Two Agents with Nested Accessible Information

TL;DR

This paper addresses a decentralized stochastic control problem with two agents sharing nested accessible information, providing structural results, a dynamic programming approach, and practical approximation methods for long-term optimal strategies.

Contribution

It introduces a structural form for optimal control strategies in a two-agent decentralized setting with shared memory, enabling scalable solutions over long horizons.

Findings

Derived a structural form for optimal strategies

Developed a dynamic programming decomposition for long horizons

Proposed simplified and approximate strategies with bounded optimality loss

Abstract

In this paper, we investigate a decentralized stochastic control problem with two agents, where a part of the memory of the second agent is also available to the first agent at each instance of time. We derive a structural form for optimal control strategies which allows us to restrict their domain to a set which does not grow in size with time. We also present a dynamic programming (DP) decomposition which can utilize our results to derive optimal strategies for arbitrarily long time horizons. Since obtaining optimal control strategies by solving this DP decomposition is computationally intensive, we present potential resolutions in the form of simplified strategies by imposing additional conditions on our model, and an approximation technique which can be used to implement our results with a bounded loss of optimality.