Attention Sequence to Sequence Model for Machine Remaining Useful Life Prediction

TL;DR

This paper introduces an attention-based sequence-to-sequence model with auxiliary tasks for more accurate and robust prediction of machinery remaining useful life, especially effective with long sensor data sequences.

Contribution

The paper proposes a novel ATS2S model that combines reconstruction and RUL prediction losses with attention mechanisms and dual-latent features, improving long-sequence modeling and prediction accuracy.

Findings

Outperforms 13 state-of-the-art methods on four datasets

Effectively handles very long sequences with attention mechanism

Achieves superior RUL prediction accuracy

Abstract

Accurate estimation of remaining useful life (RUL) of industrial equipment can enable advanced maintenance schedules, increase equipment availability and reduce operational costs. However, existing deep learning methods for RUL prediction are not completely successful due to the following two reasons. First, relying on a single objective function to estimate the RUL will limit the learned representations and thus affect the prediction accuracy. Second, while longer sequences are more informative for modelling the sensor dynamics of equipment, existing methods are less effective to deal with very long sequences, as they mainly focus on the latest information. To address these two problems, we develop a novel attention-based sequence to sequence with auxiliary task (ATS2S) model. In particular, our model jointly optimizes both reconstruction loss to empower our model with predictive capabilities (by predicting next input sequence given current input sequence) and RUL prediction loss to minimize the difference between the predicted RUL and actual RUL. Furthermore, to better handle longer sequence, we employ the attention mechanism to focus on all the important input information during training process. Finally, we propose a new dual-latent feature representation to integrate the encoder features and decoder hidden states, to capture rich semantic information in data. We conduct extensive experiments on four real datasets to evaluate the efficacy of the proposed method. Experimental results show that our proposed method can achieve superior performance over 13 state-of-the-art methods consistently.