Robustness Evaluation of Machine Learning Models for Fault Classification and Localization In Power System Protection

TL;DR

This paper presents a systematic framework for evaluating the robustness of machine learning models used in power system fault classification and localization, considering realistic sensor degradation scenarios.

Contribution

It introduces a unified methodology for benchmarking ML model robustness in power protection, including high-fidelity simulations of sensor failures and data loss.

Findings

Fault classification remains stable under most conditions but drops by 13% with single-phase loss.

Fault localization is highly sensitive, with voltage loss increasing error by over 150%.

Provides actionable insights for designing more resilient ML-based protection systems.

Abstract

The growing penetration of renewable and distributed generation is transforming power systems and challenging conventional protection schemes that rely on fixed settings and local measurements. Machine learning (ML) offers a data-driven alternative for centralized fault classification (FC) and fault localization (FL), enabling faster and more adaptive decision-making. However, practical deployment critically depends on robustness. Protection algorithms must remain reliable even when confronted with missing, noisy, or degraded sensor data. This work introduces a unified framework for systematically evaluating the robustness of ML models in power system protection. High-fidelity EMT simulations are used to model realistic degradation scenarios, including sensor outages, reduced sampling rates, and transient communication losses. The framework provides a consistent methodology for benchmarking models, quantifying the impact of limited observability, and identifying critical measurement channels required for resilient operation. Results show that FC remains highly stable under most degradation types but drops by about 13% under single-phase loss, while FL is more sensitive overall, with voltage loss increasing localization error by over 150%. These findings offer actionable guidance for robustness-aware design of future ML-assisted protection systems.