An Interpretable Probabilistic Model for Short-Term Solar Power Forecasting Using Natural Gradient Boosting

TL;DR

This paper introduces an interpretable probabilistic forecasting framework for short-term solar power prediction, combining natural gradient boosting with SHAP explanations to enhance accuracy, transparency, and physical interpretability.

Contribution

The paper presents a novel two-stage probabilistic forecasting model that integrates NGBoost and SHAP for transparent, accurate, and physically meaningful solar power predictions.

Findings

Significant improvement in forecast accuracy over state-of-the-art methods

The model reveals physical and logical relationships in data

Enhanced interpretability through SHAP explanations

Abstract

PV power forecasting models are predominantly based on machine learning algorithms which do not provide any insight into or explanation about their predictions (black boxes). Therefore, their direct implementation in environments where transparency is required, and the trust associated with their predictions may be questioned. To this end, we propose a two stage probabilistic forecasting framework able to generate highly accurate, reliable, and sharp forecasts yet offering full transparency on both the point forecasts and the prediction intervals (PIs). In the first stage, we exploit natural gradient boosting (NGBoost) for yielding probabilistic forecasts, while in the second stage, we calculate the Shapley additive explanation (SHAP) values in order to fully comprehend why a prediction was made. To highlight the performance and the applicability of the proposed framework, real data from two PV parks located in Southern Germany are employed. Comparative results with two state-of-the-art algorithms, namely Gaussian process and lower upper bound estimation, manifest a significant increase in the point forecast accuracy and in the overall probabilistic performance. Most importantly, a detailed analysis of the model's complex nonlinear relationships and interaction effects between the various features is presented. This allows interpreting the model, identifying some learned physical properties, explaining individual predictions, reducing the computational requirements for the training without jeopardizing the model accuracy, detecting possible bugs, and gaining trust in the model. Finally, we conclude that the model was able to develop complex nonlinear relationships which follow known physical properties as well as human logic and intuition.