A numerical study of 21-cm signal suppression and noise increase in direction-dependent calibration of LOFAR data

TL;DR

This study examines how direction-dependent calibration affects 21-cm signal detection in LOFAR data, demonstrating that regularised algorithms can reduce overfitting and improve signal recovery in Epoch of Reionization experiments.

Contribution

It introduces a regularised calibration method using consensus optimisation to mitigate overfitting and excess variance in LOFAR 21-cm signal analysis.

Findings

Overfitting causes excess variance on short baselines.

Regularisation reduces excess power by about a factor of 4.

Calibration approach is supported by theoretical bias-variance analysis.

Abstract

We investigate systematic effects in direction dependent gain calibration in the context of the Low-Frequency Array (LOFAR) 21-cm Epoch of Reionization (EoR) experiment. The LOFAR EoR Key Science Project aims to detect the 21-cm signal of neutral hydrogen on interferometric baselines of $50-250 \lambda$. We show that suppression of faint signals can effectively be avoided by calibrating these short baselines using only the longer baselines. However, this approach causes an excess variance on the short baselines due to small gain errors induced by overfitting during calibration. We apply a regularised expectation-maximisation algorithm with consensus optimisation (sagecal-co) to real data with simulated signals to show that overfitting can be largely mitigated by penalising spectrally non-smooth gain solutions during calibration. This reduces the excess power with about a factor 4 in the simulations. Our results agree with earlier theoretical analysis of this bias-variance trade off and support the gain-calibration approach to the LOFAR 21-cm signal data.