Interpretable Hidden Markov Model-Based Deep Reinforcement Learning Hierarchical Framework for Predictive Maintenance of Turbofan Engines

TL;DR

This paper introduces a hierarchical framework combining hidden Markov models and deep reinforcement learning to improve interpretability and decision-making in predictive maintenance of turbofan engines.

Contribution

It presents a novel hierarchical modeling approach that enhances interpretability and performance in maintenance decision policies for turbofan engines.

Findings

Outperforms baseline deep reinforcement learning methods on raw data.

Provides interpretable root cause analysis of failures.

Achieves comparable performance to prior work with added interpretability.

Abstract

An open research question in deep reinforcement learning is how to focus the policy learning of key decisions within a sparse domain. This paper emphasizes combining the advantages of inputoutput hidden Markov models and reinforcement learning towards interpretable maintenance decisions. We propose a novel hierarchical-modeling methodology that, at a high level, detects and interprets the root cause of a failure as well as the health degradation of the turbofan engine, while, at a low level, it provides the optimal replacement policy. It outperforms the baseline performance of deep reinforcement learning methods applied directly to the raw data or when using a hidden Markov model without such a specialized hierarchy. It also provides comparable performance to prior work, however, with the additional benefit of interpretability.