Integrating the Expected Future in Load Forecasts with Contextually Enhanced Transformer Models

TL;DR

This paper introduces contextually-enhanced transformer models for energy load forecasting, effectively integrating planning information to significantly reduce forecasting errors in railway and building energy applications.

Contribution

It presents a novel sequence-to-sequence transformer framework that incorporates dynamic contextual data, improving accuracy over existing methods in energy forecasting tasks.

Findings

26.6% reduction in mean absolute error for railway energy forecasting

56.3% reduction in mean absolute error for building energy forecasting

Outperforms state-of-the-art models in accuracy and flexibility

Abstract

Accurate and reliable energy forecasting is essential for power grid operators who strive to minimize extreme forecasting errors that pose significant operational challenges and incur high intra-day trading costs. Incorporating planning information -- such as anticipated user behavior, scheduled events or timetables -- provides substantial contextual information to enhance forecast accuracy and reduce the occurrence of large forecasting errors. Existing approaches, however, lack the flexibility to effectively integrate both dynamic, forward-looking contextual inputs and historical data. In this work, we conceptualize forecasting as a combined forecasting-regression task, formulated as a sequence-to-sequence prediction problem, and introduce contextually-enhanced transformer models designed to leverage all contextual information effectively. We demonstrate the effectiveness of our approach through a primary case study on nationwide railway energy consumption forecasting, where integrating contextual information into transformer models, particularly timetable data, resulted in a significant average mean absolute error reduction of 26.6%. An auxiliary case study on building energy forecasting, leveraging planned office occupancy data, further illustrates the generalizability of our method, showing an average reduction of 56.3% in mean absolute error. Compared to other state-of-the-art methods, our approach consistently outperforms existing models, underscoring the value of context-aware deep learning techniques in energy forecasting applications.