Verification of the NOAA Space Weather Prediction Center solar flare forecast (1998-2024)

TL;DR

This study rigorously evaluates NOAA's solar flare forecasts over 26 years, revealing significant shortcomings in accuracy and calibration, and underscores the need for modern, data-driven prediction models.

Contribution

It provides the first comprehensive verification of NOAA's flare forecasts, comparing them to multiple baselines and highlighting areas for improvement.

Findings

NOAA forecasts do not outperform simple baselines.

Forecasts exhibit calibration issues and high false alarm rates.

The study establishes a standard for future forecast evaluation.

Abstract

The NOAA Space Weather Prediction Center (SWPC) issues the official U.S. government forecast for M-class and X-class solar flares, yet the skill of these forecasts has never been comprehensively verified. In this study, we evaluate the SWPC probabilistic flare forecasts over a 26-year period (1998-2024), comparing them to several zero-cost and statistical baselines including persistence, climatology, Naive Bayes, and logistic regression. We find that the SWPC model does not outperform these baselines across key classification and probabilistic metrics and exhibits severe calibration issues and high false alarm rates, especially in high-stakes scenarios such as detecting the first flare after extended quiet periods. These findings demonstrate the need for more accurate and reliable eruption forecasting models which we suggest should be based on modern data-driven methods. The findings also provide a standard against which any proposed eruption prediction system should be compared. We suggest that space weather forecasters regularly update and publish analyses like the one demonstrated here to provide up-to-date standards of accuracy and reliability against which to compare new methods.