rPICARD: A CASA-based Calibration Pipeline for VLBI Data

TL;DR

rPICARD is a CASA-based calibration pipeline designed for VLBI data, offering scriptable, flexible, and scalable calibration solutions for multiple arrays, improving efficiency and control over the calibration process.

Contribution

It introduces a new CASA-based VLBI calibration pipeline with parallel processing capabilities and support for multiple arrays, enhancing flexibility and scalability over existing tools.

Findings

Parallel processing reduces calibration time significantly.

Supports multiple VLBI arrays including EHT, GMVA, VLBA, EVN.

Provides full control and scripting capabilities for calibration workflows.

Abstract

Currently, HOPS and AIPS are the primary choices for the time-consuming process of (millimeter) Very Long Baseline Interferometry (VLBI) data calibration. However, for a full end-to-end pipeline, they either lack the ability to perform easily scriptable incremental calibration or do not provide full control over the workflow with the ability to manipulate and edit calibration solutions directly. The Common Astronomy Software Application (CASA) offers all these abilities, together with a secure development future and an intuitive Python interface, which is very attractive for young radio astronomers. Inspired by the recent addition of a global fringe-fitter, the capability to convert FITS-IDI files to measurement sets, and amplitude calibration routines based on ANTAB metadata, we have developed the the CASA-based Radboud PIpeline for the Calibration of high Angular Resolution Data (rPICARD). The pipeline will be able to handle data from multiple arrays: EHT, GMVA, VLBA and the EVN in the first release. Polarization and phase-referencing calibration are supported and a spectral line mode will be added in the future. The large bandwidths of future radio observatories ask for a scalable reduction software. Within CASA, a message passing interface (MPI) implementation is used for parallelization, reducing the total time needed for processing. The most significant gain is obtained for the time-consuming fringe-fitting task where each scan be processed in parallel.

Full text

1 Introduction

The Astronomical Image Processing System (AIPS) [2] has been established as the de-facto standard calibration tool for Very Long Baseline Interferometry (VLBI) data. However, the AIPS software support is diminishing, while its successor, the Common Astronomy Software Application (CASA) [5] (formally AIPS++), became the main software for the reduction of connected interferometry data from the Very Large Array (VLA) [8] and the Atacama Large Millimeter/sub-

millimeter Array (ALMA) [10]. Thanks to an initiative from the BlackHoleCam [1] project in collaboration with the Joint Institute for VLBI ERIC, CASA has recently received a VLBI upgrade; new VLBI functionalities like a Schwab-Cotton global fringe-fitter [7] and ANTAB-based a-priori calibration routines have been added to the software suite [9].

The newer CASA software offers some advantages over AIPS, making it a more attractive choice as a future VLBI data reduction package:

The IPython user interface significantly reduces the learning curve for the new ‘Python’ generation of radio astronomers [6]. 2. 2.

CASA was designed to support batch processing. The internal data structure allows full control over all data reduction steps. 3. 3.

Parallel processing is supported, allowing the computational time to scale with available CPU power – an important feature for future observations with large bandwidths or a wide field of view.

The aforementioned CASA features are ideal prerequisites for a pipeline-based data reduction. We have developed a modular, MPI-parallelized, VLBI calibration pipeline based on CASA, called the Radboud PIpeline for the Calibration of high Angular Resolution Data (rPICARD). rPICARD v1.0.0 is able to handle data in FITS-IDI111 See https://fits.gsfc.nasa.gov/registry/fitsidi/AIPSMEM114.PDF for a description of the fits-idi data format. or Measurement Set (MS)222See https://casa.nrao.edu/Memos/229.html for a description of the current Measurement Set format. format. Simple polarization calibration and phase-referencing are possible in the first release and spectral line data calibration will be added soon. The purpose of the pipeline is to provide science-ready visibility data and thereby make VLBI more accessible to non-experts in the community. Every step of the pipeline is well documented and produces verbose diagnostic output, such that users should understand how and if the data calibration worked. It should be noted that a robust open-source VLBI calibration pipeline will bolster the reproducibility of scientific results.

rPICARD is hosted on https://bitbucket.org/M_Janssen/Picard. A specific CASA branch that contains the latest versions of the VLBI calibration functionalities is used. This branch is also open-source; a link to the latest version is given in the README file of the rPICARD repository. Eventually, this VLBI branch will be merged into the CASA master branch.

2 The CASA Calibration Framework

The CASA calibration framework is a mature and well-tested tool kit, where all radio interferometric calibration functionalities required to calibrate ALMA and VLA data are in place. With the recent additions of a fringe-fitter and special amplitude calibration tasks [9], CASA’s capabilities have been extended to the field of VLBI.

CASA’s calibration routines are based on the Hamaker-Bregman-Sault Measurement Equation [3] to calibrate full Stokes complex visibilities. Data corruption effects are corrected with incremental calibration tasks. The incremental on-the-fly application of previously obtained amplitude calibration solutions can be used to adjust the weights within the calibration solution intervals. Measurements with high system temperatures are down-weighted in this process for example. When available, CASA will make use of per-channel weights.

CASA stores visibilities and metadata as binary tables in the MS data structure. Raw FITS-IDI visibility data can be converted to a MS with the CASA importfitsidi task.

3 rPICARD Calibration

Figure 1 shows an overview of the rPICARD calibration flow. After an initial data flagging, the accor task is used to correct for erroneous recorder sampler thresholds and the scalar bandpass correction is performed based on the auto-correlations. This is done with a custom Python script with the use of basic CASA tools. The amplitude calibration is done based on standard ANTAB metadata, which is used to compute the time- and frequency-dependent system equivalent flux densities based on the gains, gain curves, and system temperatures of each station. The necessary scripts for this a-priori calibration can be found on https://github.com/jive-vlbi/casa-vlbi. Where necessary, an additional correction for the signal attenuation from the atmospheric opacity can be performed by rPICARD.

The CASA fringefit task is used for the phase calibration. To correct for intra-scan atmospheric effects at short wavelengths, optimal solution intervals are determined as a compromise between the atmospheric coherence time and the minimum required integration time to obtain fringe detections. This automated solution interval estimation is one of the reasons why rPICARD works for many different arrays without manual parameter fine-tuning. Additionally, different antennas can be calibrated on different timescales in the same scan, meaning long integration times can be used for low signal-to-noise ratio (SNR) baselines, while phases distortions on high SNR baselines can be calibrated on short timescales. The bright calibrator sources are processed first to solve for instrumental phase and delay offsets between frequency bands and – if the SNR is sufficient – for phase bandpass responses. After these instrumental effects have been calibrated out, the science targets are fringe-fitted to obtain a maximum number of detections.

The rPICARD calibration philosophy is that every calibration step can be fine-tuned, while the default input parameters will already provide high quality data for all supported arrays. The pipeline can be used as a very versatile calibration framework thanks to numerous configuration options, detailed in [4]. rPICARD produces many diagnostics for each run that can be used to judge the quality of the calibration. For example, plots of calibration solutions, lists of flags that were applied to the data, and log files with detailed information about every step of the pipeline. Simple command line arguments can be used to re-run individual steps of the pipeline. The idea is that users typically calibrate their datasets in a first pass with the default input parameters. Inspection of all diagnostic output should reveal the characteristics of the dataset and show which calibration steps did not produce sufficient solutions. Based on this information the user may compile a list of flagging statements identifying bad (un-calibratable) data. A text file containing the flagging commands will be read and applied to the data automatically by the pipeline. Next, parameters may be adjusted to improve the calibration (e.g., fringe search windows or the set of calibrator sources used to correct for the various instrumental effects).

An advantage of CASA over older calibration packages like AIPS is the built-in Message Passing Interface (MPI) scalability with CPU cores. In this framework, rPICARD will partition the measurement set such that each scan can be processed in parallel. Especially for the fringe-fitting steps, this yields a significant speed-up in computational time. The CPU scalability will be crucial for the calibration of future experiments with very large bandwidths and wide-field observations.

4 rPICARD Imaging

Traditional CLEAN imaging of VLBI datasets is often a tedious and interactive process, where large gain uncertainties and low SNR measurements require multiple rounds of careful self-calibration iterations, data flagging, and placements of CLEAN windows.

A ‘semi-automated’ imager has been implemented in rPICARD, independently of the data calibration framework. The imager is able to image data from measurement sets or UVFITS files. In essence, it is a wrapper around the CASA tclean imaging function, performing incremental imaging and self-calibration loops in an automated way. With tclean, advanced imaging methods such as multi-scale multi-frequency synthesis reconstruction methods and CLEAN auto-boxing are available. The automated placement of CLEAN boxes was developed for ALMA data. For VLBI data, the algorithm is able to recover bright and compact source features, while faint extended emission will often not be captured. Therefore, manual CLEAN boxes placement is still needed in many cases. The scripted self-calibration loops and automatic storage of used CLEAN boxes and flags are nonetheless very valuable for the reproducibility of scientific results.

More details about the rPICARD calibration and imaging strategies can be found in [4].

Acknowledgements

This work is supported by the ERC Synergy Grant ”BlackHoleCam: Imaging the Event Horizon of Black Holes” (Grant 610058).