Time-domain deep learning filtering of structured atmospheric noise for ground-based millimeter astronomy

TL;DR

This paper presents a deep learning approach using LSTM networks to effectively remove structured atmospheric noise from ground-based millimeter astronomy data, improving signal clarity over traditional methods.

Contribution

It introduces a novel LSTM-based neural network architecture trained with simulated atmospheric noise to enhance noise filtering in astronomical observations, outperforming traditional matched filtering techniques.

Findings

Neural network significantly improves SNR over raw data.

Training with incremental complexity yields robust performance.

Model effectively removes structured atmospheric noise.

Abstract

The complex physics involved in atmospheric turbulence makes it very difficult for ground-based astronomy to build accurate scintillation models and develop efficient methodologies to remove this highly structured noise from valuable astronomical observations. We argue that a Deep Learning approach can bring a significant advance to treat this problem because of deep neural networks' inherent ability to abstract non-linear patterns over a broad scale range. We propose an architecture composed of long-short term memory cells and an incremental training strategy inspired by transfer and curriculum learning. We develop a scintillation model and employ an empirical method to generate a vast catalog of atmospheric noise realizations and train the network with representative data. We face two complexity axes: the signal-to-noise ratio (SNR) and the degree of structure in the noise. Hence, we train our recurrent network to recognize simulated astrophysical point-like sources embedded in three structured noise levels, with a raw-data SNR ranging from 3 to 0.1. We find that a slow and repetitive increase in complexity is crucial during training to obtain a robust and stable learning rate that can transfer information through different data contexts. We probe our recurrent model with synthetic observational data, designing alongside a calibration methodology for flux measurements. Furthermore, we implement a traditional matched filtering (MF) to compare its performance with our neural network, finding that our final trained network can successfully clean structured noise and significantly enhance the SNR compared to raw data and in a more robust way than traditional MF.