A Semi-Decentralized Approach to Multiagent Control

TL;DR

This paper introduces a semi-decentralized framework for multiagent control under communication uncertainty, unifying existing models and providing an optimal policy algorithm evaluated on benchmarks and scenarios.

Contribution

It extends semi-Markov control to semi-decentralization in POMDPs, unifies multiple multiagent models, and presents an exact algorithm for optimal policies.

Findings

RS-SDA* efficiently computes optimal policies.

The framework unifies various multiagent communication models.

Experimental results demonstrate effectiveness on benchmarks.

Abstract

We introduce an expressive framework and algorithms for the semi-decentralized control of cooperative agents in environments with communication uncertainty. Whereas semi-Markov control admits a distribution over time for agent actions, semi-Markov communication, or what we refer to as semi-decentralization, gives a distribution over time for what actions and observations agents can store in their histories. We extend semi-decentralization to the partially observable Markov decision process (POMDP). The resulting SDec-POMDP unifies decentralized and multiagent POMDPs and several existing explicit communication mechanisms. We present recursive small-step semi-decentralized A* (RS-SDA*), an exact algorithm for generating optimal SDec-POMDP policies. RS-SDA* is evaluated on semi-decentralized versions of several standard benchmarks and a maritime medical evacuation scenario. This paper provides a well-defined theoretical foundation for exploring many classes of multiagent communication problems through the lens of semi-decentralization.