Scaling Long-Horizon Online POMDP Planning via Rapid State Space Sampling

TL;DR

This paper introduces ROP-RaS3, a fast sampling-based online POMDP solver that effectively handles long-horizon problems by generating macro actions to improve planning efficiency and quality.

Contribution

The paper presents a novel sampling-based method, ROP-RaS3, that enables scalable long-horizon POMDP planning by efficiently generating macro actions online.

Findings

ROP-RaS3 outperforms state-of-the-art methods on long-horizon POMDPs.

Successfully plans over 100-step horizons in complex environments.

Handles high-dimensional state spaces with improved efficiency.

Abstract

Partially Observable Markov Decision Processes (POMDPs) are a general and principled framework for motion planning under uncertainty. Despite tremendous improvement in the scalability of POMDP solvers, long-horizon POMDPs (e.g., $\geq15$ steps) remain difficult to solve. This paper proposes a new approximate online POMDP solver, called Reference-Based Online POMDP Planning via Rapid State Space Sampling (ROP-RaS3). ROP-RaS3 uses novel extremely fast sampling-based motion planning techniques to sample the state space and generate a diverse set of macro actions online which are then used to bias belief-space sampling and infer high-quality policies without requiring exhaustive enumeration of the action space -- a fundamental constraint for modern online POMDP solvers. ROP-RaS3 is evaluated on various long-horizon POMDPs, including on a problem with a planning horizon of more than 100 steps and a problem with a 15-dimensional state space that requires more than 20 look ahead steps. In all of these problems, ROP-RaS3 substantially outperforms other state-of-the-art methods by up to multiple folds.