Safety aware model-based reinforcement learning for optimal control of a class of output-feedback nonlinear systems

TL;DR

This paper introduces a safe model-based reinforcement learning method for nonlinear systems with output feedback, employing a novel dynamic state estimator to enable safe learning and control when full state information is unavailable.

Contribution

It develops a new output-feedback safe reinforcement learning approach using a dynamic state estimator, extending safety guarantees to partially observable nonlinear systems.

Findings

Successfully integrates a dynamic state estimator with safe RL.

Enables safe control in systems with partial observability.

Extends barrier-based safety methods to output-feedback scenarios.

Abstract

The ability to learn and execute optimal control policies safely is critical to realization of complex autonomy, especially where task restarts are not available and/or the systems are safety-critical. Safety requirements are often expressed in terms of state and/or control constraints. Methods such as barrier transformation and control barrier functions have been successfully used, in conjunction with model-based reinforcement learning, for safe learning in systems under state constraints, to learn the optimal control policy. However, existing barrier-based safe learning methods rely on full state feedback. In this paper, an output-feedback safe model-based reinforcement learning technique is developed that utilizes a novel dynamic state estimator to implement simultaneous learning and control for a class of safety-critical systems with partially observable state.