Distributed Data-Processing Pipeline for Mingantu Ultrawide Spectral Radioheliograph
F. Wang, Y. Mei, H. Deng, C.Y. Liu, D.H. Liu, S.L. Wei, W. Dai, B., Liang, Y.B. Liu, X.L. Zhang, and K.F. Ji

TL;DR
This paper presents a high-performance distributed data processing pipeline for the Chinese Spectral RadioHeliograph, enabling efficient handling of massive observational data for high-resolution solar imaging.
Contribution
It introduces a scalable, robust, and automatic data processing system built on OpenCluster for CSRH, improving data handling and processing efficiency.
Findings
Pipeline meets performance requirements for CSRH data processing
System architecture is comprehensive and adaptable
Supports real-time monitoring and high-resolution imaging
Abstract
The Chinese Spectral RadioHeliograph (CSRH) is a synthetic aperture radio interferometer built in Inner Mongolia, China. As a solar-dedicated interferometric array, CSRH is capable of producing high quality radio images at frequency range from 400 MHz to 15 GHz with high temporal, spatial, and spectral resolution.To implement high cadence imaging at wide-band and obtain more than 2 order higher multiple frequencies, the implementation of the data processing system for CSRH is a great challenge. It is urgent to build a pipeline for processing massive data of CSRH generated every day. In this paper, we develop a high performance distributed data processing pipeline (DDPP) built on the OpenCluster infrastructure for processing CSRH observational data including data storage, archiving, preprocessing, image reconstruction, deconvolution, and real-time monitoring. We comprehensively elaborate…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRadio Astronomy Observations and Technology · Geophysics and Gravity Measurements · Soil Moisture and Remote Sensing
