Comparison of Model Predictive and Reinforcement Learning Methods for Fault Tolerant Control

TL;DR

This paper compares reinforcement learning-based adaptive fault-tolerant control schemes with model predictive control for a C-130 fuel tank model, highlighting RL's robustness under faults and noise.

Contribution

It introduces two novel hierarchical reinforcement learning-based fault-tolerant control schemes and evaluates their performance against traditional model predictive control.

Findings

RL controllers outperform MPC under faults and noise

Reinforcement learning offers better adaptability in fault conditions

Controllers tested on realistic aircraft fuel tank model

Abstract

A desirable property in fault-tolerant controllers is adaptability to system changes as they evolve during systems operations. An adaptive controller does not require optimal control policies to be enumerated for possible faults. Instead it can approximate one in real-time. We present two adaptive fault-tolerant control schemes for a discrete time system based on hierarchical reinforcement learning. We compare their performance against a model predictive controller in presence of sensor noise and persistent faults. The controllers are tested on a fuel tank model of a C-130 plane. Our experiments demonstrate that reinforcement learning-based controllers perform more robustly than model predictive controllers under faults, partially observable system models, and varying sensor noise levels.