An Evaluation of Representation Learning Methods in Particle Physics Foundation Models

TL;DR

This paper systematically compares various representation learning objectives for particle physics foundation models using a unified transformer-based framework, highlighting their strengths, limitations, and establishing reproducible baselines.

Contribution

It provides a controlled, comprehensive evaluation of different learning objectives in particle physics, introducing architectural modifications that achieve state-of-the-art results.

Findings

Contrastive learning shows strong performance in jet classification.

Masked particle modeling offers complementary benefits.

Architectural tweaks improve baseline performance significantly.

Abstract

We present a systematic evaluation of representation learning objectives for particle physics within a unified framework. Our study employs a shared transformer-based particle-cloud encoder with standardized preprocessing, matched sampling, and a consistent evaluation protocol on a jet classification dataset. We compare contrastive (supervised and self-supervised), masked particle modeling, and generative reconstruction objectives under a common training regimen. In addition, we introduce targeted supervised architectural modifications that achieve state-of-the-art performance on benchmark evaluations. This controlled comparison isolates the contributions of the learning objective, highlights their respective strengths and limitations, and provides reproducible baselines. We position this work as a reference point for the future development of foundation models in particle physics, enabling more transparent and robust progress across the community.