High Performance W-stacking for Imaging Radio Astronomy Data: a Parallel and Accelerated Solution

TL;DR

This paper presents a high-performance, scalable, and portable solution for processing large radio astronomy datasets using modern HPC systems, enabling rapid imaging of billion-pixel images.

Contribution

The authors developed a parallel and accelerated imaging pipeline that efficiently scales on diverse HPC architectures, supporting datasets of any size with high speed and portability.

Findings

Supports datasets of any size compatible with hardware

Scales efficiently to thousands of cores and hundreds of GPUs

Generates billion-pixel images in under one hour

Abstract

Current and upcoming radio-interferometers are expected to produce volumes of data of increasing size that need to be processed in order to generate the corresponding sky brightness distributions through imaging. This represents an outstanding computational challenge, especially when large fields of view and/or high resolution observations are processed. We have investigated the adoption of modern High Performance Computing systems specifically addressing the gridding, FFT-transform and w-correction of imaging, combining parallel and accelerated solutions. We have demonstrated that the code we have developed can support dataset and images of any size compatible with the available hardware, efficiently scaling up to thousands of cores or hundreds of GPUs, keeping the time to solution below one hour even when images of the size of the order of billion or tens of billion of pixels are generated. In addition, portability has been targeted as a primary objective, both in terms of usability on different computing platforms and in terms of performance. The presented results have been obtained on two different state-of-the-art High Performance Computing architectures.