# A Stochastic LBFGS Algorithm for Radio Interferometric Calibration

## TL;DR

This paper introduces a stochastic LBFGS algorithm tailored for large-scale radio interferometric data calibration, enabling fine-resolution processing that preserves transient signals like FRBs and improving neural network training.

## Contribution

The paper presents a novel stochastic LBFGS algorithm capable of calibrating large datasets at high resolution without data averaging, enhancing detection of narrowband signals.

## Key findings

- Effective calibration at fine time and frequency resolutions.
- Preservation of signals like fast radio bursts (FRBs).
- Improved neural network training performance.

## Abstract

We present a stochastic, limited-memory Broyden Fletcher Goldfarb Shanno (LBFGS) algorithm that is suitable for handling very large amounts of data. A direct application of this algorithm is radio interferometric calibration of raw data at fine time and frequency resolution. Almost all existing radio interferometric calibration algorithms assume that it is possible to fit the dataset being calibrated into memory. Therefore, the raw data is averaged in time and frequency to reduce its size by many orders of magnitude before calibration is performed. However, this averaging is detrimental for the detection of some signals of interest that have narrow bandwidth and time duration such as fast radio bursts (FRBs). Using the proposed algorithm, it is possible to calibrate data at such a fine resolution that they cannot be entirely loaded into memory, thus preserving such signals. As an additional demonstration, we use the proposed algorithm for training deep neural networks and compare the performance against the mainstream first order optimization algorithms that are used in deep learning.