Transformer Training Strategies for Forecasting Multiple Load Time Series

TL;DR

This paper demonstrates that training Transformer models with a global transfer learning strategy significantly improves load forecasting accuracy across multiple clients in smart grids, outperforming other models and strategies.

Contribution

It introduces a global transfer learning approach for Transformer load forecasting models, showing its superiority over local and multivariate strategies in smart grid applications.

Findings

Global training strategy reduces forecasting errors by up to 21.8%.

Transformers outperform linear models, MLPs, and LSTMs in load forecasting.

Global strategy is effective across horizons from one day to one month.

Abstract

In the smart grid of the future, accurate load forecasts on the level of individual clients can help to balance supply and demand locally and to prevent grid outages. While the number of monitored clients will increase with the ongoing smart meter rollout, the amount of data per client will always be limited. We evaluate whether a Transformer load forecasting model benefits from a transfer learning strategy, where a global univariate model is trained on the load time series from multiple clients. In experiments with two datasets containing load time series from several hundred clients, we find that the global training strategy is superior to the multivariate and local training strategies used in related work. On average, the global training strategy results in 21.8% and 12.8% lower forecasting errors than the two other strategies, measured across forecasting horizons from one day to one month into the future. A comparison to linear models, multi-layer perceptrons and LSTMs shows that Transformers are effective for load forecasting when they are trained with the global training strategy.