Probabilistic Solar Proxy Forecasting with Neural Network Ensembles

TL;DR

This paper introduces neural network ensemble methods, including MLPs and LSTMs, to improve the accuracy of $F_{10.7 cm}$ solar proxy forecasts, demonstrating significant error reduction and uncertainty quantification.

Contribution

The work develops ensemble neural network models that outperform existing linear algorithms in solar proxy forecasting, incorporating data manipulation and multi-step predictions.

Findings

Ensemble methods improved relative MSE by 45-55%.

Models provided less biased predictions at high solar activity.

Uncertainty quantification was achieved through forecast distribution analysis.

Abstract

Space weather indices are used commonly to drive forecasts of thermosphere density, which directly affects objects in low-Earth orbit (LEO) through atmospheric drag. One of the most commonly used space weather proxies, $F_{10.7 cm}$, correlates well with solar extreme ultra-violet (EUV) energy deposition into the thermosphere. Currently, the USAF contracts Space Environment Technologies (SET), which uses a linear algorithm to forecast $F_{10.7 cm}$. In this work, we introduce methods using neural network ensembles with multi-layer perceptrons (MLPs) and long-short term memory (LSTMs) to improve on the SET predictions. We make predictions only from historical $F_{10.7 cm}$ values, but also investigate data manipulation to improve forecasting. We investigate data manipulation methods (backwards averaging and lookback) as well as multi step and dynamic forecasting. This work shows an improvement over the baseline when using ensemble methods. The best models found in this work are ensemble approaches using multi step or a combination of multi step and dynamic predictions. Nearly all approaches offer an improvement, with the best models improving between 45 and 55\% on relative MSE. Other relative error metrics were shown to improve greatly when ensembles methods were used. We were also able to leverage the ensemble approach to provide a distribution of predicted values; allowing an investigation into forecast uncertainty. Our work found models that produced less biased predictions at elevated and high solar activity levels. Uncertainty was also investigated through the use of a calibration error score metric (CES), our best ensemble reached similar CES as other work.