Vectorized Online POMDP Planning

TL;DR

VOPP is a highly parallelized online POMDP solver that significantly improves efficiency by vectorizing planning computations, enabling near-optimal solutions with much less planning time.

Contribution

The paper introduces VOPP, a novel tensor-based, fully vectorized online POMDP planner that eliminates synchronization bottlenecks and greatly enhances parallel computation efficiency.

Findings

VOPP is at least 20 times faster than existing parallel solvers.

VOPP outperforms state-of-the-art sequential solvers with 1000 times less planning budget.

VOPP achieves near-optimal solutions with high computational efficiency.

Abstract

Planning under partial observability is an essential capability of autonomous robots. The Partially Observable Markov Decision Process (POMDP) provides a powerful framework for planning under partial observability problems, capturing the stochastic effects of actions and the limited information available through noisy observations. POMDP solving could benefit tremendously from massive parallelization on today's hardware, but parallelizing POMDP solvers has been challenging. Most solvers rely on interleaving numerical optimization over actions with the estimation of their values, which creates dependencies and synchronization bottlenecks between parallel processes that can offset the benefits of parallelization. In this paper, we propose Vectorized Online POMDP Planner (VOPP), a novel parallel online solver that leverages a recent POMDP formulation which analytically solves part of the optimization component, leaving numerical computations to consist of only estimation of expectations. VOPP represents all data structures related to planning as a collection of tensors, and implements all planning steps as fully vectorized computations over this representation. The result is a massively parallel online solver with no dependencies or synchronization bottlenecks between concurrent processes. Experimental results indicate that VOPP is at least $20\times$ more efficient in computing near-optimal solutions compared to an existing state-of-the-art parallel online solver. Moreover, VOPP outperforms state-of-the-art sequential online solvers, while using a planning budget that is $1000\times$ smaller.