Developing a Real Time Processing System for HERA
Paul La Plante, Peter K. G. Williams, Joshua S. Dillon

TL;DR
This paper presents the development of a real-time processing system for HERA, a large radio telescope array, enabling efficient data quality inspection, flagging, and initial calibration to handle over 60 TB of nightly data.
Contribution
It introduces a modular, open-source real-time processing system with pipeline management and data handling components tailored for HERA's data volume and analysis needs.
Findings
Successfully processes over 60 TB of data nightly
Provides initial calibration and data quality flagging
Designed for adaptability to other radio astronomy projects
Abstract
The Hydrogen Epoch of Reionization Array (HERA) is a radio telescope in the Karoo desert of South Africa endeavoring to observe Cosmic Dawn and the Epoch of Reionization. When fully constructed, it will consist of 350 antennas and generate over 60 terabytes (TB) of data each night. In order to keep pace with the relatively large rate of data, we have developed the real-time processing (RTP) system for HERA. The RTP is responsible for inspecting data for acceptable levels of quality and flagging unusable data, as well as providing an initial calibration solution. The RTP system consists of the pipeline management system, called the hera_opm package, and the data management system, called the librarian. Though the systems have been developed in the context of analyzing HERA data, they feature public and open source software, and have been designed to be adapted and used in other contexts…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRadio Astronomy Observations and Technology · Astronomy and Astrophysical Research · Computational Physics and Python Applications
