Approximate Dec-POMDP Solving Using Multi-Agent A*

TL;DR

This paper introduces an A*-based algorithm for approximate Dec-POMDP solving that improves scalability and performance for larger horizons by employing clustering, pruning, and novel heuristics, with competitive results.

Contribution

It presents a scalable A*-based method for approximate Dec-POMDP solutions using clustering, pruning, and new heuristics, including an upper bound algorithm for long horizons.

Findings

Competitive performance to state-of-the-art methods

Superior performance on multiple benchmarks

Effective upper bound computation for long horizons

Abstract

We present an A*-based algorithm to compute policies for finite-horizon Dec-POMDPs. Our goal is to sacrifice optimality in favor of scalability for larger horizons. The main ingredients of our approach are (1) using clustered sliding window memory, (2) pruning the A* search tree, and (3) using novel A* heuristics. Our experiments show competitive performance to the state-of-the-art. Moreover, for multiple benchmarks, we achieve superior performance. In addition, we provide an A* algorithm that finds upper bounds for the optimum, tailored towards problems with long horizons. The main ingredient is a new heuristic that periodically reveals the state, thereby limiting the number of reachable beliefs. Our experiments demonstrate the efficacy and scalability of the approach.