Generalizing Fault Detection Against Domain Shifts Using Stratification-Aware Cross-Validation

TL;DR

This paper explores how ensemble learning can improve detection of subtle incipient anomalies under domain shifts, addressing challenges of limited early-stage anomaly data and proposing strategies for more effective models.

Contribution

It introduces methods to leverage ensemble uncertainty for identifying incipient anomalies and discusses design improvements for ensemble models in this context.

Findings

Ensemble methods improve incipient anomaly detection performance.

Uncertainty from ensembles helps identify potential misclassifications.

The paper highlights pitfalls and proposes strategies for better ensemble design.

Abstract

Incipient anomalies present milder symptoms compared to severe ones, and are more difficult to detect and diagnose due to their close resemblance to normal operating conditions. The lack of incipient anomaly examples in the training data can pose severe risks to anomaly detection methods that are built upon Machine Learning (ML) techniques, because these anomalies can be easily mistaken as normal operating conditions. To address this challenge, we propose to utilize the uncertainty information available from ensemble learning to identify potential misclassified incipient anomalies. We show in this paper that ensemble learning methods can give improved performance on incipient anomalies and identify common pitfalls in these models through extensive experiments on two real-world datasets. Then, we discuss how to design more effective ensemble models for detecting incipient anomalies.