Testing the Efficacy of Hyperparameter Optimization Algorithms in Short-Term Load Forecasting

TL;DR

This paper evaluates five hyperparameter optimization algorithms for short-term load forecasting using XGBoost on the Panama dataset, highlighting runtime efficiency and accuracy differences across methods.

Contribution

It compares multiple HPO algorithms for STLF, providing insights into their performance and statistical significance, which was previously underexplored.

Findings

HPO algorithms outperform Random Search in runtime

Bayesian optimization had the lowest accuracy in univariate models

Significant performance differences confirmed by statistical tests

Abstract

Accurate forecasting of electrical demand is essential for maintaining a stable and reliable power grid, optimizing the allocation of energy resources, and promoting efficient energy consumption practices. This study investigates the effectiveness of five hyperparameter optimization (HPO) algorithms -- Random Search, Covariance Matrix Adaptation Evolution Strategy (CMA--ES), Bayesian Optimization, Partial Swarm Optimization (PSO), and Nevergrad Optimizer (NGOpt) across univariate and multivariate Short-Term Load Forecasting (STLF) tasks. Using the Panama Electricity dataset (n=48,049), we evaluate HPO algorithms' performances on a surrogate forecasting algorithm, XGBoost, in terms of accuracy (i.e., MAPE, $R^2$) and runtime. Performance plots visualize these metrics across varying sample sizes from 1,000 to 20,000, and Kruskal--Wallis tests assess the statistical significance of the performance differences. Results reveal significant runtime advantages for HPO algorithms over Random Search. In univariate models, Bayesian optimization exhibited the lowest accuracy among the tested methods. This study provides valuable insights for optimizing XGBoost in the STLF context and identifies areas for future research.