PerFormer: A Permutation Based Vision Transformer for Remaining Useful Life Prediction

TL;DR

This paper introduces PerFormer, a permutation-based vision transformer that enhances remaining useful life prediction by transforming multivariate sensor data into a format suitable for ViT, outperforming existing CNN and RNN methods.

Contribution

The paper proposes a novel permutation-based ViT approach with a permutation loss function to improve RUL prediction from sensor data.

Findings

PerFormer outperforms CNNs, RNNs, and other Transformer models on NASA's C-MAPSS dataset.

The permutation loss effectively guides the model to learn suitable data permutations.

Results demonstrate significant accuracy improvements in RUL prediction.

Abstract

Accurately estimating the remaining useful life (RUL) for degradation systems is crucial in modern prognostic and health management (PHM). Convolutional Neural Networks (CNNs), initially developed for tasks like image and video recognition, have proven highly effectively in RUL prediction, demonstrating remarkable performance. However, with the emergence of the Vision Transformer (ViT), a Transformer model tailored for computer vision tasks such as image classification, and its demonstrated superiority over CNNs, there is a natural inclination to explore its potential in enhancing RUL prediction accuracy. Nonetheless, applying ViT directly to multivariate sensor data for RUL prediction poses challenges, primarily due to the ambiguous nature of spatial information in time series data. To address this issue, we introduce the PerFormer, a permutation-based vision transformer approach designed to permute multivariate time series data, mimicking spatial characteristics akin to image data, thereby making it suitable for ViT. To generate the desired permutation matrix, we introduce a novel permutation loss function aimed at guiding the convergence of any matrix towards a permutation matrix. Our experiments on NASA's C-MAPSS dataset demonstrate the PerFormer's superior performance in RUL prediction compared to state-of-the-art methods employing CNNs, Recurrent Neural Networks (RNNs), and various Transformer models. This underscores its effectiveness and potential in PHM applications.