A Strong Duality Result for Constrained POMDPs with Multiple Cooperative Agents

TL;DR

This paper establishes a strong duality result for decentralized constrained POMDPs with multiple cooperative agents, under specific conditions on observations, actions, and costs, advancing the theoretical understanding of such systems.

Contribution

It extends duality theory to decentralized constrained POMDPs with multiple agents, using advanced measure convergence and minimax theorems, a novel theoretical development.

Findings

Strong duality holds for infinite-horizon discounted costs under specified conditions.

The results apply to systems with countable observations and finite actions.

Theoretical framework facilitates future algorithm development for decentralized POMDPs.

Abstract

The work studies the problem of decentralized constrained POMDPs in a team-setting where multiple nonstrategic agents have asymmetric information. Using an extension of Sion's Minimax theorem for functions with positive infinity and results on weak-convergence of measures, strong duality is established for the setting of infinite-horizon expected total discounted costs when the observations lie in a countable space, the actions are chosen from a finite space, the constraint costs are bounded, and the objective cost is bounded from below.