LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detection

TL;DR

This paper introduces a novel framework combining Large Language Models, reinforcement learning, and unsupervised techniques to improve time series anomaly detection, especially under limited labeled data conditions.

Contribution

It presents a unified approach integrating LLM-based rewards, VAE-enhanced scaling, and active learning for more effective anomaly detection in time series data.

Findings

Achieves state-of-the-art accuracy on Yahoo-A1 and SMD benchmarks.

Operates effectively with limited labeled data.

Demonstrates robustness in data-constrained environments.

Abstract

Detecting anomalies in time series data is crucial for finance, healthcare, sensor networks, and industrial monitoring applications. However, time series anomaly detection often suffers from sparse labels, complex temporal patterns, and costly expert annotation. We propose a unified framework that integrates Large Language Model (LLM)-based potential functions for reward shaping with Reinforcement Learning (RL), Variational Autoencoder (VAE)-enhanced dynamic reward scaling, and active learning with label propagation. An LSTM-based RL agent leverages LLM-derived semantic rewards to guide exploration, while VAE reconstruction errors add unsupervised anomaly signals. Active learning selects the most uncertain samples, and label propagation efficiently expands labeled data. Evaluations on Yahoo-A1 and SMD benchmarks demonstrate that our method achieves state-of-the-art detection accuracy under limited labeling budgets and operates effectively in data-constrained settings. This study highlights the promise of combining LLMs with RL and advanced unsupervised techniques for robust, scalable anomaly detection in real-world applications.