Feature Selection on a Flare Forecasting Testbed: A Comparative Study of 24 Methods

TL;DR

This study systematically compares 24 feature selection methods on a solar flare forecasting benchmark dataset, evaluating their effectiveness in identifying relevant features for improved predictive performance.

Contribution

First comprehensive comparison of various feature selection algorithms applied to solar flare forecasting using the SWAN-SF dataset.

Findings

Identified the most effective feature selection methods for flare prediction

Provided insights into feature relevance and model performance

Established a reproducible pipeline for future research

Abstract

The Space-Weather ANalytics for Solar Flares (SWAN-SF) is a multivariate time series benchmark dataset recently created to serve the heliophysics community as a testbed for solar flare forecasting models. SWAN-SF contains 54 unique features, with 24 quantitative features computed from the photospheric magnetic field maps of active regions, describing their precedent flare activity. In this study, for the first time, we systematically attacked the problem of quantifying the relevance of these features to the ambitious task of flare forecasting. We implemented an end-to-end pipeline for preprocessing, feature selection, and evaluation phases. We incorporated 24 Feature Subset Selection (FSS) algorithms, including multivariate and univariate, supervised and unsupervised, wrappers and filters. We methodologically compared the results of different FSS algorithms, both on the multivariate time series and vectorized formats, and tested their correlation and reliability, to the extent possible, by using the selected features for flare forecasting on unseen data, in univariate and multivariate fashions. We concluded our investigation with a report of the best FSS methods in terms of their top-k features, and the analysis of the findings. We wish the reproducibility of our study and the availability of the data allow the future attempts be comparable with our findings and themselves.