Big Data Challenges of FAST
Youling Yue, Di Li
TL;DR
This paper discusses the significant big-data challenges faced by FAST, a large radio telescope, focusing on its high data rate and volume, and presents solutions for pulsar search data processing.
Contribution
It identifies the unique data challenges of FAST and proposes specific solutions for managing pulsar search data at high rates.
Findings
FAST data stream exceeds 6 GB/s in rate
Annual data volume reaches about 20 PB
Proposed solutions address pulsar search data challenges
Abstract
We present the big-data challenges posed by the science operation of the Five-hundred-meter Aperture Spherical radio Telescope (FAST). Unlike the common usage of the word `big-data', which tend to emphasize both quantity and diversity, the main characteristics of FAST data stream is its single-source data rate at more than 6 GB/s and the resulting data volume at about 20 PB per year. We describe here the main culprit of such a high data rate and large volume, namely pulsar search, and our solution.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.