Bumblebee: Foundation Model for Particle Physics Discovery

TL;DR

Bumblebee is a versatile foundation model inspired by BERT, designed for particle physics discovery, capturing complex data features and improving various analysis tasks with significant accuracy gains.

Contribution

It introduces a novel particle physics foundation model that removes positional encodings and embeds particle 4-vectors, enhancing task performance and discovery potential.

Findings

Improves dileptonic top quark reconstruction resolution by 10-20%.

Achieves high AUROC scores in toponium discrimination and initial state classification.

Demonstrates flexibility for diverse particle physics applications.

Abstract

Bumblebee is a foundation model for particle physics discovery, inspired by BERT. By removing positional encodings and embedding particle 4-vectors, Bumblebee captures both generator- and reconstruction-level information while ensuring sequence-order invariance. Pre-trained on a masked task, it improves dileptonic top quark reconstruction resolution by 10-20% and excels in downstream tasks, including toponium discrimination (AUROC 0.877) and initial state classification (AUROC 0.625). The flexibility of Bumblebee makes it suitable for a wide range of particle physics applications, especially the discovery of new particles.