The Black-Box Simplex Architecture for Runtime Assurance of Autonomous CPS

TL;DR

This paper introduces a black-box runtime assurance framework for autonomous cyber-physical systems that ensures safety by switching control authority without needing to verify the internal workings of controllers.

Contribution

It proposes the Black-Box Simplex Architecture, enabling safety guarantees for complex controllers like neural networks without requiring their internal verification.

Findings

Proves the safety of the architecture.

Demonstrates safe multi-robot coordination with model-predictive control.

Shows neural networks can prevent collisions despite unsafe outputs.

Abstract

The Simplex Architecture is a runtime assurance framework where control authority may switch from an unverified and potentially unsafe advanced controller to a backup baseline controller in order to maintain the safety of an autonomous cyber-physical system. In this work, we show that runtime checks can replace the requirement to statically verify safety of the baseline controller. This is important as there are many powerful control techniques, such as model-predictive control and neural network controllers, that work well in practice but are difficult to statically verify. Since the method does not use internal information about the advanced or baseline controller, we call the approach the Black-Box Simplex Architecture. We prove the architecture is safe and present two case studies where (i) model-predictive control provides safe multi-robot coordination, and (ii) neural networks provably prevent collisions in groups of F-16 aircraft, despite the controllers occasionally outputting unsafe commands.