Efficient Multiagent Planning via Shared Action Suggestions

TL;DR

This paper introduces a scalable multiagent planning method that uses shared action suggestions to efficiently prune beliefs, bridging POMDPs and Dec-POMDPs, and achieving near-centralized performance.

Contribution

It proposes a novel approach that communicates only suggested joint actions, reducing complexity while maintaining high performance in multiagent decision making.

Findings

Performance comparable to centralized methods on benchmarks

Reduces computational complexity by solving individual POMDPs

Enables effective belief pruning through shared actions

Abstract

Decentralized partially observable Markov decision processes with communication (Dec-POMDP-Com) provide a framework for multiagent decision making under uncertainty, but the NEXP-complete complexity for finite-horizon problems renders solutions intractable in general. While sharing actions and observations can reduce the complexity to PSPACE-complete, we propose an approach that bridges POMDPs and Dec-POMDPs by communicating only suggested joint actions, eliminating the need to share observations while retaining near-centralized performance. Our algorithm estimates joint beliefs using shared actions to prune infeasible beliefs. Each agent maintains possible belief sets for other agents, pruning them based on suggested actions to form an estimated joint belief usable with any centralized policy. This approach requires solving a POMDP for each agent, reducing computational complexity while preserving performance. We demonstrate its effectiveness on several Dec-POMDP benchmarks, showing performance comparable to centralized methods when shared actions enable effective belief pruning. This action-based communication framework offers a natural avenue for integrating human-agent cooperation, opening new directions for scalable multiagent planning under uncertainty, with applications in both autonomous systems and human-agent teams.