ES-dRNN with Dynamic Attention for Short-Term Load Forecasting

TL;DR

This paper introduces an enhanced hybrid neural network model with dynamic attention mechanisms for improved short-term load forecasting, effectively capturing complex seasonal patterns and variances in time series data.

Contribution

It presents a novel gated recurrent cell with attention for better input selection, combined with existing techniques, leading to significant accuracy improvements in load forecasting.

Findings

Outperforms existing statistical and machine learning models in accuracy

Demonstrates effectiveness across 35 European countries

Introduces a new attentive dilated recurrent cell

Abstract

Short-term load forecasting (STLF) is a challenging problem due to the complex nature of the time series expressing multiple seasonality and varying variance. This paper proposes an extension of a hybrid forecasting model combining exponential smoothing and dilated recurrent neural network (ES-dRNN) with a mechanism for dynamic attention. We propose a new gated recurrent cell -- attentive dilated recurrent cell, which implements an attention mechanism for dynamic weighting of input vector components. The most relevant components are assigned greater weights, which are subsequently dynamically fine-tuned. This attention mechanism helps the model to select input information and, along with other mechanisms implemented in ES-dRNN, such as adaptive time series processing, cross-learning, and multiple dilation, leads to a significant improvement in accuracy when compared to well-established statistical and state-of-the-art machine learning forecasting models. This was confirmed in the extensive experimental study concerning STLF for 35 European countries.