A Closer Look at Bearing Fault Classification Approaches

TL;DR

This paper examines how different data handling and evaluation choices impact the performance of machine learning models for bearing fault classification, emphasizing the importance of proper experimental design for real-world applications.

Contribution

It highlights the effects of data partitioning, labeling methods, and evaluation metrics on model performance, providing guidelines for more reliable fault diagnosis models.

Findings

Data from the same bearing should not be split across training and testing to avoid over-optimistic results.

PCA-based labeling effectively generates failure labels in run-to-failure experiments.

F scores provide more meaningful evaluation for unbalanced real-world failure data.

Abstract

Rolling bearing fault diagnosis has garnered increased attention in recent years owing to its presence in rotating machinery across various industries, and an ever increasing demand for efficient operations. Prompt detection and accurate prediction of bearing failures can help reduce the likelihood of unexpected machine downtime and enhance maintenance schedules, averting lost productivity. Recent technological advances have enabled monitoring the health of these assets at scale using a variety of sensors, and predicting the failures using modern Machine Learning (ML) approaches including deep learning architectures. Vibration data has been collected using accelerated run-to-failure of overloaded bearings, or by introducing known failure in bearings, under a variety of operating conditions such as rotating speed, load on the bearing, type of bearing fault, and data acquisition frequency. However, in the development of bearing failure classification models using vibration data there is a lack of consensus in the metrics used to evaluate the models, data partitions used to evaluate models, and methods used to generate failure labels in run-to-failure experiments. An understanding of the impact of these choices is important to reliably develop models, and deploy them in practical settings. In this work, we demonstrate the significance of these choices on the performance of the models using publicly-available vibration datasets, and suggest model development considerations for real world scenarios. Our experimental findings demonstrate that assigning vibration data from a given bearing across training and evaluation splits leads to over-optimistic performance estimates, PCA-based approach is able to robustly generate labels for failure classification in run-to-failure experiments, and $F$ scores are more insightful to evaluate the models with unbalanced real-world failure data.