Stream Processing for Solar Physics: Applications and Implications for Big Solar Data

TL;DR

This paper reviews stream mining techniques from computer science and explores their potential application to analyze the massive data generated by solar observatories like SDO, highlighting opportunities and challenges.

Contribution

It provides an overview of stream mining algorithms and discusses their adaptation for processing and analyzing large-scale solar physics data sets.

Findings

Many existing stream mining methods can be adapted for solar data analysis.

No single solution currently addresses all challenges of solar data stream mining.

Applying these methods could significantly enhance solar data analysis capabilities.

Abstract

Modern advances in space technology have enabled the capture and recording of unprecedented volumes of data. In the field of solar physics this is most readily apparent with the advent of the Solar Dynamics Observatory (SDO), which returns in excess of 1 terabyte of data daily. While we now have sufficient capability to capture, transmit and store this information, the solar physics community now faces the new challenge of analysis and mining of high-volume and potentially boundless data sets such as this: a task known to the computer science community as stream mining. In this paper, we survey existing and established stream mining methods in the context of solar physics, with a goal of providing an introductory overview of stream mining algorithms employed by the computer science fields. We consider key concepts surrounding stream mining that are applicable to solar physics, outlining existing algorithms developed to address this problem in other fields of study, and discuss their applicability to massive solar data sets. We also discuss the considerations and trade-offs that may need to be made when applying stream mining methods to solar data. We find that while no one single solution is readily available, many of the methods now employed in other data streaming applications could successfully be modified to apply to solar data and prove invaluable for successful analysis and mining of this new source.