Scaling performance of the SAGECal calibration package: from LOFAR to SKA

TL;DR

This paper evaluates how the SAGECal calibration software scales with increasing data size and station count, crucial for SKA's future large-scale radio observations, by analyzing its algorithms and runtime performance.

Contribution

It provides an analysis of SAGECal's scaling behavior and algorithms, demonstrating its suitability for large-scale SKA data processing.

Findings

SAGECal runtime scales approximately linearly with the number of stations.

Algorithms inside SAGECal explain its scaling behavior.

Preparation for SKA1 LOW requires efficient calibration software.

Abstract

This decade, the Square Kilometre Array (SKA) will perform its first observations. Preparations for building dishes, antennas, correlators and infrastructure are well underway. Concurrently, software for the processing of SKA observations is developed at a number of levels. At a more basic level there are the telescope monitoring and control systems and also the correlator software. On top of that, in order to deliver science ready data products, software pipelines are needed for radio frequency interference (RFI) mitigation, averaging, calibration and imaging. Here, we focus on the SAGECal calibration package, in particular on the times needed to obtain calibration solutions. This is an important aspect, since this package is now used for the Epoch of Reionization (EoR) Key Science Project of LOFAR, but will also have to run optimally on SKA1 LOW. In terms of number of stations used for observing this amounts to a factor 10 increase, from 51 to 512 stations. Consequentially, the disk space needed to store an observation will increase by a factor 100, provided the number of frequency channels remains the same. In this paper we investigate the scaling behaviour of SAGECal, whose runtimes should ideally scale linearly with the number of stations. We also explain the algorithms inside SAGECal and use them to explain its scaling behaviour.