Simplex-enabled Safe Continual Learning Machine

TL;DR

This paper introduces the SeC-Learning Machine, a novel safe continual learning framework for autonomous systems that combines Simplex logic and physics-regulated deep reinforcement learning to ensure safety and adaptability in real-world applications.

Contribution

It presents a new architecture integrating high-performance learning, high-assurance safety, and dynamic switching to guarantee safety during continual learning in safety-critical systems.

Findings

Ensures lifetime safety in continual learning scenarios.

Addresses the Sim2Real gap effectively.

Demonstrates successful experiments on cart-pole and quadruped robot.

Abstract

This paper proposes the SeC-Learning Machine: Simplex-enabled safe continual learning for safety-critical autonomous systems. The SeC-learning machine is built on Simplex logic (that is, ``using simplicity to control complexity'') and physics-regulated deep reinforcement learning (Phy-DRL). The SeC-learning machine thus constitutes HP (high performance)-Student, HA (high assurance)-Teacher, and Coordinator. Specifically, the HP-Student is a pre-trained high-performance but not fully verified Phy-DRL, continuing to learn in a real plant to tune the action policy to be safe. In contrast, the HA-Teacher is a mission-reduced, physics-model-based, and verified design. As a complementary, HA-Teacher has two missions: backing up safety and correcting unsafe learning. The Coordinator triggers the interaction and the switch between HP-Student and HA-Teacher. Powered by the three interactive components, the SeC-learning machine can i) assure lifetime safety (i.e., safety guarantee in any continual-learning stage, regardless of HP-Student's success or convergence), ii) address the Sim2Real gap, and iii) learn to tolerate unknown unknowns in real plants. The experiments on a cart-pole system and a real quadruped robot demonstrate the distinguished features of the SeC-learning machine, compared with continual learning built on state-of-the-art safe DRL frameworks with approaches to addressing the Sim2Real gap.