Fault-Tolerant Control of Degrading Systems with On-Policy Reinforcement Learning

TL;DR

This paper introduces an adaptive reinforcement learning control method for fault-tolerant management of degrading systems, eliminating the need for fault detection and diagnosis, and ensuring stable learning through combined online and offline training.

Contribution

It presents a novel on-policy reinforcement learning approach that adapts to system degradation without prior fault knowledge, integrating online and offline learning for improved stability and efficiency.

Findings

Effective fault-tolerant control demonstrated on aircraft fuel system

Stable learning achieved without fault detection step

Online and offline learning integration improves adaptation

Abstract

We propose a novel adaptive reinforcement learning control approach for fault tolerant control of degrading systems that is not preceded by a fault detection and diagnosis step. Therefore, \textit{a priori} knowledge of faults that may occur in the system is not required. The adaptive scheme combines online and offline learning of the on-policy control method to improve exploration and sample efficiency, while guaranteeing stable learning. The offline learning phase is performed using a data-driven model of the system, which is frequently updated to track the system's operating conditions. We conduct experiments on an aircraft fuel transfer system to demonstrate the effectiveness of our approach.