Fault Identification Enhancement with Reinforcement Learning (FIERL)

TL;DR

This paper introduces FIERL, a reinforcement learning-based method that enhances active fault detection by optimizing control strategies without needing detailed passive detector models, applicable to complex fault scenarios.

Contribution

FIERL is a novel simulation-based reinforcement learning approach that designs control inputs for fault detection, applicable to complex scenarios and passive detector types without requiring internal detector knowledge.

Findings

FIERL effectively generalizes to unseen fault dynamics.

It demonstrates robustness in actuator fault diagnosis.

Applicable to complex fault mode scenarios.

Abstract

This letter presents a novel approach in the field of Active Fault Detection (AFD), by explicitly separating the task into two parts: Passive Fault Detection (PFD) and control input design. This formulation is very general, and most existing AFD literature can be viewed through this lens. By recognizing this separation, PFD methods can be leveraged to provide components that make efficient use of the available information, while the control input is designed in order to optimize the gathering of information. The core contribution of this work is FIERL, a general simulation-based approach for the design of such control strategies, using Constrained Reinforcement Learning (CRL) to optimize the performance of arbitrary passive detectors. The control policy is learned without the need of knowing the passive detector inner workings, making FIERL broadly applicable. However, it is especially useful when paired with the design of an efficient passive component. Unlike most AFD approaches, FIERL can handle fairly complex scenarios such as continuous sets of fault modes. The effectiveness of FIERL is tested on a benchmark problem for actuator fault diagnosis, where FIERL is shown to be fairly robust, being able to generalize to fault dynamics not seen in training.