Sparse tree search optimality guarantees in POMDPs with continuous observation spaces

TL;DR

This paper provides a theoretical guarantee for a sampling-based algorithm in POMDPs with continuous observations, showing it can approximate optimal solutions with sufficient computation.

Contribution

It introduces and proves the effectiveness of the POWSS algorithm, offering formal guarantees for its accuracy and near-optimality in continuous observation POMDPs.

Findings

POWSS estimates Q-values accurately with high probability.

Increasing computational resources improves near-optimal performance.

Provides formal theoretical justification for observation likelihood weighting techniques.

Abstract

Partially observable Markov decision processes (POMDPs) with continuous state and observation spaces have powerful flexibility for representing real-world decision and control problems but are notoriously difficult to solve. Recent online sampling-based algorithms that use observation likelihood weighting have shown unprecedented effectiveness in domains with continuous observation spaces. However there has been no formal theoretical justification for this technique. This work offers such a justification, proving that a simplified algorithm, partially observable weighted sparse sampling (POWSS), will estimate Q-values accurately with high probability and can be made to perform arbitrarily near the optimal solution by increasing computational power.