Path Integral Control in Gaussian Belief Space for Partially Observed Systems

TL;DR

This paper develops a Gaussian belief space extension of path integral control for partially observed systems, deriving conditions for linearization and introducing the MPPI-Belief algorithm, validated through navigation experiments.

Contribution

It formulates path integral control in Gaussian belief space, derives exact linearization conditions, and proposes a new MPPI-Belief algorithm for partially observed stochastic control.

Findings

MPPI-Belief outperforms certainty-equivalent and particle-filter baselines in navigation tasks.

Exact Cole-Hopf linearization is achieved in Gaussian belief space under certain conditions.

The approach effectively manages state-dependent observation noise in control problems.

Abstract

This paper extends path integral control (PIC) to partially observed systems by formulating the problem in Gaussian belief space. PIC relies on the diffusion being proportional to the control channel -- the so-called matching condition -- to linearize the Hamilton-Jacobi-Bellman equation via the Cole-Hopf transform; we show that this condition fails in infinite-dimensional belief space under non-affine observations. Restricting to Gaussian beliefs yields a finite-dimensional approximation with deterministic covariance evolution, reducing the problem to stochastic control of the belief mean. We derive necessary and sufficient conditions for matching in this reduced space, obtain an exact Cole-Hopf linearization with a Feynman-Kac representation, and develop the MPPI-Belief algorithm. Numerical experiments on a navigation task with state-dependent observation noise demonstrate the effectiveness of MPPI-Belief relative to certainty-equivalent and particle-filter-based baselines.