Predictive Maintenance for General Aviation Using Convolutional Transformers

TL;DR

This paper introduces a new challenging multivariate time series dataset for aircraft maintenance classification, proposes a convolutional transformer model that outperforms RNNs, and demonstrates effective augmentation techniques to enhance model generalization.

Contribution

The work presents the NGAFID-MC dataset as a new benchmark and introduces Conv-MHSA, a novel convolutional transformer architecture for improved MTS classification in aviation.

Findings

Conv-MHSA outperforms RNNs in classification accuracy

Image-inspired augmentations reduce overfitting

Models integrated into NGAFID for real-time maintenance detection

Abstract

Predictive maintenance systems have the potential to significantly reduce costs for maintaining aircraft fleets as well as provide improved safety by detecting maintenance issues before they come severe. However, the development of such systems has been limited due to a lack of publicly labeled multivariate time series (MTS) sensor data. MTS classification has advanced greatly over the past decade, but there is a lack of sufficiently challenging benchmarks for new methods. This work introduces the NGAFID Maintenance Classification (NGAFID-MC) dataset as a novel benchmark in terms of difficulty, number of samples, and sequence length. NGAFID-MC consists of over 7,500 labeled flights, representing over 11,500 hours of per second flight data recorder readings of 23 sensor parameters. Using this benchmark, we demonstrate that Recurrent Neural Network (RNN) methods are not well suited for capturing temporally distant relationships and propose a new architecture called Convolutional Multiheaded Self Attention (Conv-MHSA) that achieves greater classification performance at greater computational efficiency. We also demonstrate that image inspired augmentations of cutout, mixup, and cutmix, can be used to reduce overfitting and improve generalization in MTS classification. Our best trained models have been incorporated back into the NGAFID to allow users to potentially detect flights that require maintenance as well as provide feedback to further expand and refine the NGAFID-MC dataset.