Load balancing for distributed interferometric image reconstruction

TL;DR

This paper introduces a scalable distributed algorithm for wide-field radio interferometric image reconstruction that efficiently handles large data sets with exact non-coplanar correction, enabling high-resolution imaging on cluster computing resources.

Contribution

It extends the distributed $w$-stacking $w$-projection algorithm with a new load-balancing degridding method, improving efficiency for large-scale interferometric imaging.

Findings

Reconstructed images of 2 billion visibilities in just over an hour.

Demonstrated the necessity of load balancing for large data sets.

Algorithm implemented in the open-source PURIFY software.

Abstract

We present a new algorithm to perform wide-field radio interferometric image reconstruction, with exact non-coplanar correction, that scales to big-data. This algorithm allows us to image 2 billion visibilities on 50 nodes of a computing cluster for a 25 by 25 degree field of view, in a little over an hour. We build on the recently developed distributed $w$-stacking $w$-projection hybrid algorithm, extending it to include a new distributed degridding algorithm that balances the computational load of the $w$-projection gridding kernels. The implementation of our algorithm is made publicly available in the PURIFY software package. Wide-field image reconstruction for data sets of this size cannot be performed effectively using the allocated computational resources without computational load balancing, demonstrating that our algorithms are critical for next-generation wide-field radio interferometers.