Attention-based Neural Network Emulators for Multi-Probe Data Vectors Part III: Modeling The Next Generation Surveys

TL;DR

This paper demonstrates that attention-based neural network emulators can accurately model CMB power spectra within the standard cosmological model, reducing outliers and enabling direct application to real data for upcoming cosmological surveys.

Contribution

It introduces attention mechanisms into neural emulators for CMB spectra, achieving higher precision and broader applicability compared to traditional architectures.

Findings

Attention-based models reduce outliers in emulating CMB spectra.

Emulators achieve less than 10% outliers with 200,000-400,000 training vectors.

Models meet precision requirements for current and future CMB experiments.

Abstract

Machine learning can accelerate cosmological inferences that involve many sequential evaluations of computationally expensive data vectors. Previous works in this series have examined how machine learning architectures impact emulator accuracy and training time for optical shear and galaxy clustering 2-point function. In this final manuscript, we explore neural network performance when emulating Cosmic Microwave Background temperature and polarization power spectra. We maximize the volume of applicability in the parameter space of our emulators within the standard $\Lambda$-cold-dark-matter model while ensuring that errors are below cosmic variance. Relative to standard multi-layer perceptron architectures, we find the dot-product-attention mechanism reduces the number of outliers among testing cosmologies, defined as the fraction of testing points with $\Delta \chi^2 > 0.2$ relative to \textsc{CAMB} outputs, for a wide range of training set sizes. Such precision enables attention-based emulators to be directly applied to real data without requiring any additional correction via importance sampling. Combined with pre-processing techniques and optimized activation and loss functions, attention-based models can meet the precision criteria set by current and future CMB and lensing experiments. For each of Planck, Simons Observatory, CMB S4, and CMB HD, we find the fraction of outlier points to be less than $10\%$ with around $2\times10^5$ to $4\times10^5$ training data vectors. We further explore the applications of these methods to supernova distance, weak lensing, and galaxy clustering, as well as alternative architectures and pre-processing techniques.