Transfer Learning Beyond the Standard Model

TL;DR

This paper explores transfer learning in cosmology, showing it can reduce simulation costs for beyond-$ mf ext{Lambda}$CDM models but also highlights challenges like negative transfer due to physical degeneracies.

Contribution

It demonstrates the effectiveness of transfer learning from the standard cosmological model to various beyond-$ mf ext{Lambda}$CDM scenarios and analyzes transfer architectures.

Findings

Pre-training on $ mf ext{Lambda}$CDM accelerates inference for other models.

Negative transfer occurs with strong degeneracies between models.

Bottleneck architectures improve transfer performance.

Abstract

Machine learning enables powerful cosmological inference but typically requires many high-fidelity simulations covering many cosmological models. Transfer learning offers a way to reduce the simulation cost by reusing knowledge across models. We show that pre-training on the standard model of cosmology, $\Lambda$CDM, and fine-tuning on various beyond-$\Lambda$CDM scenarios -- including massive neutrinos, modified gravity, and primordial non-Gaussianities -- can enable inference with significantly fewer beyond-$\Lambda$CDM simulations. However, we also show that negative transfer can occur when strong physical degeneracies exist between $\Lambda$CDM and beyond-$\Lambda$CDM parameters. We consider various transfer architectures, finding that including bottleneck structures provides the best performance. Our findings illustrate the opportunities and pitfalls of foundation-model approaches in physics: pre-training can accelerate inference, but may also hinder learning new physics.