NICE^k Metrics: Unified and Multidimensional Framework for Evaluating Deterministic Solar Forecasting Accuracy

TL;DR

The paper introduces the NICE^k metrics, a unified, multidimensional framework for evaluating deterministic solar forecasting accuracy, addressing limitations of traditional error metrics by capturing different error aspects and improving discriminative power.

Contribution

The NICE^k framework provides a novel, interpretable, and statistically robust set of metrics for solar forecast evaluation, outperforming traditional methods in significance and generalizability.

Findings

NICE^k metrics outperform traditional error metrics in statistical significance.

NICE2 and NICESigma show higher discriminative power in evaluations.

Framework validated with data from 68 Spanish solar stations.

Abstract

Accurate solar energy output prediction is key for integrating renewables into grids, maintaining stability, and improving energy management. However, standard error metrics such as Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Skill Scores (SS) fail to capture the multidimensional nature of solar irradiance forecasting. These metrics lack sensitivity to forecastability, rely on arbitrary baselines (e.g., clear-sky models), and are poorly suited for operational use. To address this, we introduce the NICEk framework (Normalized Informed Comparison of Errors, with k = 1, 2, 3, Sigma), offering a robust and interpretable evaluation of forecasting models. Each NICEk score corresponds to an Lk norm: NICE1 targets average errors, NICE2 emphasizes large deviations, NICE3 highlights outliers, and NICESigma combines all. Using Monte Carlo simulations and data from 68 stations in the Spanish SIAR network, we evaluated methods including autoregressive models, extreme learning, and smart persistence. Theoretical and empirical results align when assumptions hold (e.g., R^2 ~ 1.0 for NICE2). Most importantly, NICESigma consistently shows higher discriminative power (p < 0.05), outperforming traditional metrics (p > 0.05). The NICEk metrics exhibit stronger statistical significance (e.g., p-values from 10^-6 to 0.004 across horizons) and greater generalizability. They offer a unified and operational alternative to standard error metrics in deterministic solar forecasting.