Data Challenges for Next-generation Radio Telescopes

TL;DR

The paper discusses upcoming data challenges in radio-astronomy driven by new technologies and large-scale surveys, emphasizing the need for advanced data processing, storage, and analysis techniques to fully exploit SKA Pathfinder data.

Contribution

It reviews the key data challenges and highlights the necessity for developing new methods to handle massive datasets and complex data analysis in next-generation radio telescopes.

Findings

Identification of petabyte-scale data storage needs.

Emphasis on developing cross-identification techniques for millions of galaxies.

Recognition of the importance of real-time data processing pipelines.

Abstract

Radio-astronomy is about to embark on a new way of doing science. The revolution that is about to take place is not due to the enormous sensitivity of the Square Kilometre Array, which is still a decade away, but due to its pathfinders, which are pioneering new ways of doing radio-astronomy. These new ways include multi-pixel phased-array feeds, the goal of producing science-ready images from a real-time pipeline processor, and from the vast amounts of survey data that will be available in the public domain soon after observing. Here I review the data challenges that need to be addressed if we are to reap all the science that potentially resides in SKA Pathfinder data. Some challenges are obvious, such as petabytes of data storage, and some are less obvious, such as the techniques we have yet to develop to perform cross- identifications on millions of galaxies.