Learning Safe, Generalizable Perception-based Hybrid Control with Certificates

TL;DR

This paper presents LOCUS, a neural network-based hybrid control system that ensures safety and goal-reaching in robotic navigation using perception data, without relying on separate state estimation.

Contribution

It introduces a novel learning-enabled hybrid controller that directly learns safety and control functions in observation space, eliminating the need for separate perception-estimation modules.

Findings

LOCUS safely navigates unknown environments

It generalizes well outside training scenarios

Successfully demonstrated in simulation and hardware

Abstract

Many robotic tasks require high-dimensional sensors such as cameras and Lidar to navigate complex environments, but developing certifiably safe feedback controllers around these sensors remains a challenging open problem, particularly when learning is involved. Previous works have proved the safety of perception-feedback controllers by separating the perception and control subsystems and making strong assumptions on the abilities of the perception subsystem. In this work, we introduce a novel learning-enabled perception-feedback hybrid controller, where we use Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) to show the safety and liveness of a full-stack perception-feedback controller. We use neural networks to learn a CBF and CLF for the full-stack system directly in the observation space of the robot, without the need to assume a separate perception-based state estimator. Our hybrid controller, called LOCUS (Learning-enabled Observation-feedback Control Using Switching), can safely navigate unknown environments, consistently reach its goal, and generalizes safely to environments outside of the training dataset. We demonstrate LOCUS in experiments both in simulation and in hardware, where it successfully navigates a changing environment using feedback from a Lidar sensor.