Perception-Based Beliefs for POMDPs with Visual Observations

TL;DR

This paper introduces Perception-based Beliefs for POMDPs (PBP), a framework that integrates perception models into belief updates to efficiently handle high-dimensional visual observations, improving robustness and performance.

Contribution

The paper proposes PBP, a novel framework that incorporates perception models into POMDP belief updates, enabling tractable planning with visual data and robustness to perception errors.

Findings

PBP outperforms existing deep RL methods in visual POMDPs.

Uncertainty quantification enhances robustness against visual corruption.

Belief updates align with standard methods when classifiers are exact.

Abstract

Partially observable Markov decision processes (POMDPs) are a principled planning model for sequential decision-making under uncertainty. Yet, real-world problems with high-dimensional observations, such as camera images, remain intractable for traditional belief- and filtering-based solvers. To tackle this problem, we introduce the Perception-based Beliefs for POMDPs framework (PBP), which complements such solvers with a perception model. This model takes the form of an image classifier which maps visual observations to probability distributions over states. PBP incorporates these distributions directly into belief updates, so the underlying solver does not need to reason explicitly over high-dimensional observation spaces. We show that the belief update of PBP coincides with the standard belief update if the image classifier is exact. Moreover, to handle classifier imprecision, we incorporate uncertainty quantification and introduce two methods to adjust the belief update accordingly. We implement PBP using two traditional POMDP solvers and empirically show that (1) it outperforms existing end-to-end deep RL methods and (2) uncertainty quantification improves robustness of PBP against visual corruption.