Dual Aspect Self-Attention based on Transformer for Remaining Useful Life Prediction

TL;DR

This paper introduces DAST, a novel transformer-based dual self-attention model for RUL prediction that effectively processes long sequences and extracts sensor and time step features simultaneously, outperforming existing methods.

Contribution

The paper proposes a purely self-attention based encoder-decoder structure with dual encoders for parallel feature extraction in RUL prediction, avoiding RNN/CNN modules.

Findings

DAST outperforms state-of-the-art RUL prediction methods on turbofan datasets.

The dual encoder design effectively captures sensor and time step features separately.

Self-attention enables better processing of long data sequences in RUL prediction.

Abstract

Remaining useful life prediction (RUL) is one of the key technologies of condition-based maintenance, which is important to maintain the reliability and safety of industrial equipments. Massive industrial measurement data has effectively improved the performance of the data-driven based RUL prediction method. While deep learning has achieved great success in RUL prediction, existing methods have difficulties in processing long sequences and extracting information from the sensor and time step aspects. In this paper, we propose Dual Aspect Self-attention based on Transformer (DAST), a novel deep RUL prediction method, which is an encoder-decoder structure purely based on self-attention without any RNN/CNN module. DAST consists of two encoders, which work in parallel to simultaneously extract features of different sensors and time steps. Solely based on self-attention, the DAST encoders are more effective in processing long data sequences, and are capable of adaptively learning to focus on more important parts of input. Moreover, the parallel feature extraction design avoids mutual influence of information from two aspects. Experiments on two widely used turbofan engines datasets show that our method significantly outperforms the state-of-the-art RUL prediction methods.