Point-Based POMDP Algorithms: Improved Analysis and Implementation

TL;DR

This paper introduces a new complexity bound for point-based POMDP algorithms that combines dimensionality and historical factors, along with improved implementation techniques for better efficiency.

Contribution

It presents a novel complexity bound using discounted reachability and discusses enhancements to heuristic search value iteration algorithms.

Findings

Derived a new complexity bound combining curse of dimensionality and history

Implemented tighter initial bounds and avoided linear programs

Enhanced efficiency through better use of sparsity

Abstract

Existing complexity bounds for point-based POMDP value iteration algorithms focus either on the curse of dimensionality or the curse of history. We derive a new bound that relies on both and uses the concept of discounted reachability; our conclusions may help guide future algorithm design. We also discuss recent improvements to our (point-based) heuristic search value iteration algorithm. Our new implementation calculates tighter initial bounds, avoids solving linear programs, and makes more effective use of sparsity.