Closing the Loop on Runtime Monitors with Fallback-Safe MPC

TL;DR

This paper introduces a safety certification framework for perception-enabled robotic systems using robust MPC and conformal prediction, enabling reliable operation in novel environments with fewer samples than retraining.

Contribution

It presents a novel combination of robust MPC and conformal prediction for certifying safety and detecting perception degradation in robotic systems deployed in new contexts.

Findings

Certifies safety with significantly fewer samples than retraining perception models.

Demonstrates safety-preserving control in simulated aircraft taxiing and quadrotor scenarios.

Provides an open-source simulation platform and visual evidence of results.

Abstract

When we rely on deep-learned models for robotic perception, we must recognize that these models may behave unreliably on inputs dissimilar from the training data, compromising the closed-loop system's safety. This raises fundamental questions on how we can assess confidence in perception systems and to what extent we can take safety-preserving actions when external environmental changes degrade our perception model's performance. Therefore, we present a framework to certify the safety of a perception-enabled system deployed in novel contexts. To do so, we leverage robust model predictive control (MPC) to control the system using the perception estimates while maintaining the feasibility of a safety-preserving fallback plan that does not rely on the perception system. In addition, we calibrate a runtime monitor using recently proposed conformal prediction techniques to certifiably detect when the perception system degrades beyond the tolerance of the MPC controller, resulting in an end-to-end safety assurance. We show that this control framework and calibration technique allows us to certify the system's safety with orders of magnitudes fewer samples than required to retrain the perception network when we deploy in a novel context on a photo-realistic aircraft taxiing simulator. Furthermore, we illustrate the safety-preserving behavior of the MPC on simulated examples of a quadrotor. We open-source our simulation platform and provide videos of our results at our project page: https://tinyurl.com/fallback-safe-mpc.