TPCNet: Representation learning for HI mapping

TL;DR

TPCNet is a neural network that combines convolutional and transformer architectures with positional encodings to accurately analyze neutral hydrogen spectra, outperforming previous CNN models in stability and accuracy.

Contribution

The paper introduces TPCNet, a novel neural network architecture that integrates CNNs and Transformers with positional encodings for HI spectral analysis, demonstrating superior performance over shallow CNNs.

Findings

TPCNet achieves 10% higher testing accuracy than deep CNNs.

The model is robust to dataset shuffling and weight initialization.

Higher spectral resolution improves model performance but increases training time.

Abstract

We introduce TPCNet, a neural network predictor that combines Convolutional and Transformer architectures with Positional encodings, for neutral atomic hydrogen (HI) spectral analysis. Trained on synthetic datasets, our models predict cold neutral gas fraction ($f_\text{CNM}$) and HI opacity correction factor ($R_\text{HI}$) from emission spectra based on the learned relationships between the desired output parameters and observables (optically-thin column density and peak brightness). As a follow-up to Murray et al. (2020)'s shallow Convolutional Neural Network (CNN), we construct deep CNN models and compare them to TPCNet models. TPCNet outperforms deep CNNs, achieving a 10% average increase in testing accuracy, algorithmic (training) stability, and convergence speed. Our findings highlight the robustness of the proposed model with sinusoidal positional encoding applied directly to the spectral input, addressing perturbations in training dataset shuffling and convolutional network weight initializations. Higher spectral resolutions with increased spectral channels offer advantages, albeit with increased training time. Diverse synthetic datasets enhance model performance and generalization, as demonstrated by producing $f_\text{CNM}$ and $R_\text{HI}$ values consistent with evaluation ground truths. Applications of TPCNet to observed emission data reveal strong agreement between the predictions and Gaussian decomposition-based estimates (from emission and absorption surveys), emphasizing its potential in HI spectral analysis.