Evaluation of Large Language Models for Numeric Anomaly Detection in Power Systems

TL;DR

This study evaluates the effectiveness of large language models, specifically GPT-OSS-20B, for numeric anomaly detection in power systems, comparing various learning approaches on the IEEE 14-bus system.

Contribution

It provides a comprehensive assessment of LLMs for numeric anomaly detection in power grids, including different training and adaptation methods, and introduces a rule-aware detection framework.

Findings

LLMs show potential for anomaly detection but have limitations.

Hybrid approaches improve detection accuracy.

The rule-aware framework enhances interpretability.

Abstract

Large language models (LLMs) have gained increasing attention in power grids for their general-purpose capabilities. Meanwhile, anomaly detection (AD) remains critical for grid resilience, requiring accurate and interpretable decisions based on multivariate telemetry. Yet the performance of LLMs on large-scale numeric data for AD remains largely unexplored. This paper presents a comprehensive evaluation of LLMs for numeric AD in power systems. We use GPT-OSS-20B as a representative model and evaluate it on the IEEE 14-bus system. A standardized prompt framework is applied across zero-shot, few-shot, in-context learning, low rank adaptation (LoRA), fine-tuning, and a hybrid LLM-traditional approach. We adopt a rule-aware design based on the three-sigma criterion, and report detection performance and rationale quality. This study lays the groundwork for further investigation into the limitations and capabilities of LLM-based AD and its integration with classical detectors in cyber-physical power grid applications.