Safety-critical Control Under Partial Observability: Reach-Avoid POMDP meets Belief Space Control

TL;DR

This paper introduces a layered belief space control architecture for reach-avoid POMDPs, enabling real-time, safe decision-making in robotics by decoupling goal, safety, and information objectives.

Contribution

It proposes modular belief space controllers using BCLFs and BCBFs, learned via reinforcement learning, to improve safety and efficiency over existing POMDP solvers.

Findings

Real-time control synthesis with lightweight quadratic programs.

Enhanced safety and task success in simulation and space-robotics experiments.

Effective handling of non-Gaussian beliefs with high-dimensional state spaces.

Abstract

Partially Observable Markov Decision Processes (POMDPs) provide a principled framework for robot decision-making under uncertainty. Solving reach-avoid POMDPs, however, requires coordinating three distinct behaviors: goal reaching, safety, and active information gathering to reduce uncertainty. Existing online POMDP solvers attempt to address all three within a single belief tree search, but this unified approach struggles with the conflicting time scales inherent to these objectives. We propose a layered, certificate-based control architecture that operates directly in belief space, decoupling goal reaching, information gathering, and safety into modular components. We introduce Belief Control Lyapunov Functions (BCLFs) that formalize information gathering as a Lyapunov convergence problem in belief space, and show how they can be learned via reinforcement learning. For safety, we develop Belief Control Barrier Functions (BCBFs) that leverage conformal prediction to provide probabilistic safety guarantees over finite horizons. The resulting control synthesis reduces to lightweight quadratic programs solvable in real time, even for non-Gaussian belief representations with dimension $>10^4$. Experiments in simulation and on a space-robotics platform demonstrate real-time performance and improved safety and task success compared to state-of-the-art constrained POMDP solvers.