FM4NPP: A Scaling Foundation Model for Nuclear and Particle Physics

TL;DR

This paper introduces FM4NPP, a large-scale foundation model for particle physics that leverages a new dataset and self-supervised training to outperform baselines and adapt efficiently across diverse tasks.

Contribution

The work presents a novel self-supervised training method, a large dataset, and demonstrates the scalability and task generalization of a foundation model in experimental particle physics.

Findings

Model with up to 188 million parameters outperforms baselines

Task-agnostic representations can be specialized with a linear mapping

Model exhibits robust data-efficient adaptation

Abstract

Large language models have revolutionized artificial intelligence by enabling large, generalizable models trained through self-supervision. This paradigm has inspired the development of scientific foundation models (FMs). However, applying this capability to experimental particle physics is challenging due to the sparse, spatially distributed nature of detector data, which differs dramatically from natural language. This work addresses if an FM for particle physics can scale and generalize across diverse tasks. We introduce a new dataset with more than 11 million particle collision events and a suite of downstream tasks and labeled data for evaluation. We propose a novel self-supervised training method for detector data and demonstrate its neural scalability with models that feature up to 188 million parameters. With frozen weights and task-specific adapters, this FM consistently outperforms baseline models across all downstream tasks. The performance also exhibits robust data-efficient adaptation. Further analysis reveals that the representations extracted by the FM are task-agnostic but can be specialized via a single linear mapping for different downstream tasks.