Benchmarking deep learning models for bearing fault diagnosis using the CWRU dataset: A multi-label approach

TL;DR

This paper introduces a multi-label fault diagnosis approach for the CWRU bearing dataset, addressing data leakage issues, class imbalance, and realism, and benchmarks several deep learning models on this improved setup.

Contribution

It proposes a multi-label formulation and a realistic dataset division to improve fault diagnosis accuracy and evaluation, along with a comprehensive benchmark of deep learning models.

Findings

Multi-label approach reduces data leakage effects.

New dataset division improves model performance.

Benchmarking reveals strengths and weaknesses of models.

Abstract

This paper proposes a novel approach for modeling the problem of fault diagnosis using the Case Western Reserve University (CWRU) bearing fault dataset. Although the dataset is considered a standard reference for testing new algorithms, the typical dataset division suffers from data leakage, as shown by Hendriks et al. (2022) and Abburi et al. (2023), leading to papers reporting over-optimistic results. While their proposed division significantly mitigates this issue, it does not eliminate it entirely. Moreover, their proposed multi-class classification task can still lead to an unrealistic scenario by excluding the possibility of more than one fault type occurring at the same or different locations. As advocated in this paper, a multi-label formulation (detecting the presence of each type of fault for each location) can solve both issues, leading to a scenario closer to reality. Additionally, this approach mitigates the heavy class imbalance of the CWRU dataset, where faulty cases appear much more frequently than healthy cases, even though the opposite is more likely to occur in practice. A multi-label formulation also enables a more precise evaluation using prevalence-independent evaluation metrics for binary classification, such as the ROC curve. Finally, this paper proposes a more realistic dataset division that allows for more diversity in the training dataset while keeping the division free from data leakage. The results show that this new division can significantly improve performance while enabling a fine-grained error analysis. As an application of our approach, a comparative benchmark is performed using several state-of-the-art deep learning models applied to 1D and 2D signal representations in time and/or frequency domains.