An Algorithm Architecture for Radio Interferometric Data Processing

TL;DR

This paper introduces a scalable, modular algorithm architecture for radio interferometric data processing, combining calibration and imaging as optimization problems, and demonstrates high-performance implementation on diverse hardware.

Contribution

It presents a unified mathematical framework and software architecture for scalable radio astronomy data processing, leveraging modern performance engineering techniques.

Findings

Achieved ~2 TB/hour processing rate using 100 GPUs.

Successfully imaged deep field data from VLA with microJy/beam noise.

Demonstrated scalability across various high-performance computing platforms.

Abstract

We present a foundational, scalable algorithm architecture for processing data from aperture synthesis radio telescopes. The analysis leading to the architecture is rooted in the theory of aperture synthesis, signal processing and numerical optimization keeping it scalable for variations in computing load, algorithmic complexity, and accommodate the continuing evolution of algorithms. It also adheres to scientific software design principles and use of modern performance engineering techniques providing a stable foundation for long-term scalability, performance, and development cost. We first show that algorithms for both calibration and imaging algorithms share a common mathematical foundation and can be expressed as numerical optimization problems. We then decompose the resulting mathematical framework into fundamental conceptual architectural components, and assemble calibration and imaging algorithms from these foundational components. For a physical architectural view, we used a library of algorithms implemented in the LibRA software for the various architectural components, and used the Kokkos framework in the compute-intensive components for performance portable implementation. This was deployed on hardware ranging from desktop-class computers to multiple super-computer class high-performance computing (HPC) and high-throughput computing (HTC) platforms with a variety of CPU and GPU architectures, and job schedulers (HTCondor and Slurm). As a test, we imaged archival data from the NSF's Karl G. Jansky Very Large Array (VLA) telescope in the A-array configuration for the Hubble Ultra Deep Field. Using over 100 GPUs we achieve a processing rate of ~2 Terabyte per hour to make one of the deepest images in the 2 -- 4 GHz band with an RMS noise of ~1 microJy/beam.