Conditional Deep Generative Models for Belief State Planning

TL;DR

This paper introduces a novel approach using conditional deep generative models to represent beliefs in high-dimensional POMDPs, improving accuracy and planning performance over traditional methods.

Contribution

The paper proposes using cDGMs for belief representation in high-dimensional POMDPs, demonstrating superior performance over particle filters.

Findings

cDGMs outperform particle filters in belief accuracy

cDGMs improve planning performance in mineral exploration POMDPs

Effective in high-dimensional, continuous state spaces

Abstract

Partially observable Markov decision processes (POMDPs) are used to model a wide range of applications, including robotics, autonomous vehicles, and subsurface problems. However, accurately representing the belief is difficult for POMDPs with high-dimensional states. In this paper, we propose a novel approach that uses conditional deep generative models (cDGMs) to represent the belief. Unlike traditional belief representations, cDGMs are well-suited for high-dimensional states and large numbers of observations, and they can generate an arbitrary number of samples from the posterior belief. We train the cDGMs on data produced by random rollout trajectories and show their effectiveness in solving a mineral exploration POMDP with a large and continuous state space. The cDGMs outperform particle filter baselines in both task-agnostic measures of belief accuracy as well as in planning performance.