Simplified POMDP Planning with an Alternative Observation Space and Formal Performance Guarantees

TL;DR

This paper introduces a novel method for simplifying POMDP planning by using an alternative observation space and formal bounds, enabling faster decision-making with guaranteed performance in uncertain environments.

Contribution

It proposes a new framework that switches to a compact observation space with formal bounds, improving planning efficiency while maintaining solution quality.

Findings

Significant speedup in POMDP planning in simulations.

Formal bounds relate simplified and original POMDP solutions.

Effective in both exact and approximate solvers.

Abstract

Online planning under uncertainty in partially observable domains is an essential capability in robotics and AI. The partially observable Markov decision process (POMDP) is a mathematically principled framework for addressing decision-making problems in this challenging setting. However, finding an optimal solution for POMDPs is computationally expensive and is feasible only for small problems. In this work, we contribute a novel method to simplify POMDPs by switching to an alternative, more compact, observation space and simplified model to speedup planning with formal performance guarantees. We introduce the notion of belief tree topology, which encodes the levels and branches in the tree that use the original and alternative observation space and models. Each belief tree topology comes with its own policy space and planning performance. Our key contribution is to derive bounds between the optimal Q-function of the original POMDP and the simplified tree defined by a given topology with a corresponding simplified policy space. These bounds are then used as an adaptation mechanism between different tree topologies until the optimal action of the original POMDP can be determined. Further, we consider a specific instantiation of our framework, where the alternative observation space and model correspond to a setting where the state is fully observable. We evaluate our approach in simulation, considering exact and approximate POMDP solvers and demonstrating a significant speedup while preserving solution quality. We believe this work opens new exciting avenues for online POMDP planning with formal performance guarantees.