NSW-EPNews: A News-Augmented Benchmark for Electricity Price Forecasting with LLMs

TL;DR

This paper introduces NSW-EPNews, a comprehensive benchmark combining time-series data, weather, and news to evaluate the performance of traditional models and large language models in electricity price forecasting.

Contribution

It presents the first multimodal benchmark dataset for electricity price prediction that includes textual news, weather, and numerical data, enabling evaluation of LLMs in this domain.

Findings

Traditional models gain little from news features.

State-of-the-art LLMs show modest performance improvements.

LLMs often hallucinate or fabricate data in forecasts.

Abstract

Electricity price forecasting is a critical component of modern energy-management systems, yet existing approaches heavily rely on numerical histories and ignore contemporaneous textual signals. We introduce NSW-EPNews, the first benchmark that jointly evaluates time-series models and large language models (LLMs) on real-world electricity-price prediction. The dataset includes over 175,000 half-hourly spot prices from New South Wales, Australia (2015-2024), daily temperature readings, and curated market-news summaries from WattClarity. We frame the task as 48-step-ahead forecasting, using multimodal input, including lagged prices, vectorized news and weather features for classical models, and prompt-engineered structured contexts for LLMs. Our datasets yields 3.6k multimodal prompt-output pairs for LLM evaluation using specific templates. Through compresive benchmark design, we identify that for traditional statistical and machine learning models, the benefits gain is marginal from news feature. For state-of-the-art LLMs, such as GPT-4o and Gemini 1.5 Pro, we observe modest performance increase while it also produce frequent hallucinations such as fabricated and malformed price sequences. NSW-EPNews provides a rigorous testbed for evaluating grounded numerical reasoning in multimodal settings, and highlights a critical gap between current LLM capabilities and the demands of high-stakes energy forecasting.