Safe Controller for Output Feedback Linear Systems using Model-Based Reinforcement Learning

TL;DR

This paper introduces a novel output-feedback safe reinforcement learning method for linear systems, enabling safe policy learning without full state feedback, demonstrated through simulation results.

Contribution

It presents a barrier-aware dynamic state estimator that allows safe reinforcement learning using output feedback, expanding applicability in real-world safety-critical systems.

Findings

Barrier transformation effectively enables online reinforcement learning.

The proposed method ensures safety during learning in simulation.

Output feedback suffices for safe control policy development.

Abstract

The objective of this research is to enable safety-critical systems to simultaneously learn and execute optimal control policies in a safe manner to achieve complex autonomy. Learning optimal policies via trial and error, i.e., traditional reinforcement learning, is difficult to implement in safety-critical systems, particularly when task restarts are unavailable. Safe model-based reinforcement learning techniques based on a barrier transformation have recently been developed to address this problem. However, these methods rely on full state feedback, limiting their usability in a real-world environment. In this work, an output-feedback safe model-based reinforcement learning technique based on a novel barrier-aware dynamic state estimator has been designed to address this issue. The developed approach facilitates simultaneous learning and execution of safe control policies for safety-critical linear systems. Simulation results indicate that barrier transformation is an effective approach to achieve online reinforcement learning in safety-critical systems using output feedback.