Machine-learning based particle-flow algorithm in CMS

TL;DR

This paper discusses recent developments in a machine learning-based particle-flow algorithm for CMS, utilizing transformer models to improve event reconstruction by directly inferring particles from detector data.

Contribution

It introduces CMS-specific training datasets, model architecture, and integration methods for the machine-learned particle flow algorithm using transformer models.

Findings

Enhanced particle reconstruction accuracy

Successful integration with CMS offline software

Potential for improved event analysis efficiency

Abstract

The particle-flow (PF) algorithm provides a global event description by reconstructing final-state particles and is central to event reconstruction in CMS. Recently, end-to-end machine learning (ML) approaches have been proposed to directly optimize physical quantities of interest and to leverage heterogeneous computing architectures. One such approach, machine-learned particle flow (MLPF), uses a transformer model to infer particles directly from tracks and clusters in a single pass. We present recent CMS developments in MLPF, including training datasets, model architecture, reconstruction metrics, and integration with offline reconstruction software.

Equations

$$L(Y,\hat{Y})=\sum _i{L}_{\mathrm{cls}-\mathrm{binary}}(y_i,\widehat{y_i})+L_{\mathrm{cls}-\mathrm{PID}}(y_i,\widehat{y_i})+L_{\mathrm{cls}-\mathrm{PU}}({\alpha }_i,\widehat{{\alpha }_i})+L_{\mathrm{reg}}(y_i,\widehat{y_i}).$$

Full text

1 Introduction

The CMS particle-flow (PF) algorithm [1] uses rule-based methods, such as proximity-based linking—associating tracks and calorimeter clusters—to reconstruct a global, particle-level view of each event. In contrast, machine-learned particle-flow (MLPF) uses transformer models trained on simulation to exploit low-level features of particle interactions with the detector that may not be immediately obvious from a first principles approach based on feature engineering. In these proceedings, we present an MLPF implementation integrated within the CMS software framework (CMSSW), trained on Monte Carlo (MC) simulation samples with pileup (PU) and validated both in simulation and on proton-proton collisions data collected during Run 3 (2022 – 2026) by the CMS experiment [2, 3].

2 Datasets and simulation samples

The CMS detector response is simulated using Geant4 (v11.2.2) [4, 5, 6]. Parton showering, fragmentation, and hadronization are modeled with pythia (v8.311) [7], using the CP5 underlying event tune [8]. Simulated MC samples for training and validation are produced at a center-of-mass energy of 14 TeV, under Run 3 (2023) detector and accelerator conditions, including $\mathrm{t}\overline{\mathrm{t}}$, QCD multijet, and all-hadronic $\mathrm{Z}\to\tau\tau$ processes. For samples with PU, we apply a flat profile with an average of 55–75 interactions per bunch crossing. For validation on data, we use a small subset of CMS Run 3 data passing the dijet triggers, collected at a center-of-mass energy of 13.6 TeV.

3 MLPF algorithm

3.1 Simulation-based target

MLPF is trained to reconstruct target particles that leave hits in the detector either directly or through their decay products, and is validated against pythia truth particles. Due to the finite granularity and thresholds in the detector simulation and reconstruction chain, there is an unavoidable smearing between the truth particles and the target particles. For example, if a particle is below the energy threshold to leave hits in the Geant4 simulation, or if it does not produce any hits that survive upstream reconstruction (e.g., track or cluster reconstruction), it is excluded from the target and thus cannot be reconstructed.

In addition, for samples generated with PU, there is an additional mismatch between the MLPF target and the truth: while the truth set contains only hard-scatter particles, the target set includes particles from both the hard interaction and PU. To handle this, we record the PU energy fraction associated with each target particle. Additionally, we define a PU-masked target, which includes only the subset of target particles associated with the hard interaction, enabling validation and performance studies without PU contamination. We validate the MLPF target definition using inclusive $\mathrm{t}\overline{\mathrm{t}}$ samples with PU, confirming that the target aligns well with pileup-subtracted pythia truth particles, as shown in Fig. 1.

3.2 MLPF architecture and loss function

The model architecture follows the original MLPF approach [9, 10, 11], replacing GNN layers with transformers accelerated with FlashAttention [12] for improved scalability and performance. The model takes as input a set of reconstructed tracks and calorimeter clusters, each represented by a feature vector $x_{i}$, collectively forming the set of PF elements $X$. The input features for each track and cluster are described in Section 3.3. The model outputs reconstructed particle candidates $\hat{y}_{i}\in\hat{Y}$, trained to match a set of simulation-based target particles $Y$. Each target particle $y_{i}$ is characterized by three quantities: (1) its particle identification (PID), (2) a binary label $\alpha_{i}$ indicating whether it originates from pileup, and (3) its four-momentum, parameterized as $(p_{\mathrm{T}},\eta,\sin{\phi},\cos{\phi},E)$.

Since the input and output sets have different cardinalities, we associate each target particle with a single unique input element, designated as the primary element, to define a per-particle loss function. This allows reconstruction of events with a variable number of particles. The assignment is defined by an injective, non-surjective function informed by physics priors: charged particles are assigned to their originating tracks, while neutral particles are assigned to the cluster where they contribute most of their energy.

We adopt a particle-level objective function following the MLPF approach from Refs. [9, 10, 11], composed of four loss terms: a particle existence binary classification loss, a multiclass PID classification loss, a binary classification loss for the PU label and a regression loss for the particle four-momentum. The model is trained to minimize a sum of per-particle loss terms over all input tracks and clusters:

[TABLE]

The particle existence binary classification loss $L_{\text{cls-binary}}$ predicts whether a track or cluster corresponds to a reconstructed particle and is implemented using binary cross entropy. The multiclass PID loss $L_{\text{cls-PID}}$ discriminates among particle types (e.g., electrons vs. photons) and is computed using focal loss [13] to address class imbalance by emphasizing harder-to-classify examples. The binary classification loss $L_{\text{cls-PU}}$, implemented using binary cross entropy, identifies reconstructed particles that arose from PU vertices. The regression loss $L_{\mathrm{reg}}$ estimates the components of the four-momentum using the mean squared error between the predicted and target values.

To improve performance at high-$p_{\mathrm{T}}$, we weight the per-particle loss terms with a $\sqrt{p_{\mathrm{T}}}$ factor for the $p_{\mathrm{T}}$ and $E$ components, placing greater emphasis on particles in the high-$p_{\mathrm{T}}$ regime. We have observed that this weight term improves the jet energy scale and resolution performance over the baseline.

The model is trained end-to-end for eight epochs on a single NVIDIA A100 80GB GPU, requiring approximately 260 hours. All loss components—binary classification, PID, PU ID, and regression—converge smoothly with no signs of overtraining. We observe no performance gains from additional training. The final model contains approximately 4 million learnable parameters.

3.3 Input features

The input features for each track include the track transverse momentum ($p_{\mathrm{T}}$) and its uncertainty, the pseudorapidity $\eta$ and azimuthal angle $\phi$ of the particle momentum and their uncertainties (both at the reference point, and extrapolated to the ECAL and HCAL), the Cartesian momentum components, the electric charge, the number of reconstructed hits in the track, and the track vertex coordinates $(v_{x},v_{y},v_{z})$. The polar angle $\theta$, computed with respect to the beam axis, and $\lambda$, defined as $\lambda=\frac{\pi}{2}-\theta$ [14], along with their uncertainties, are also included.

For calorimeter clusters, the input features include the transverse energy ($E_{\mathrm{T}}$), $\eta$, $\phi$, electromagnetic calorimeter energy ($E_{\mathrm{ECAL}}$), hadronic calorimeter energy ($E_{\mathrm{HCAL}}$), corrected energy and its associated uncertainty, time and its uncertainty, Cartesian spatial coordinates ($x$, $y$, $z$). Additional features include the number of hits in the cluster and the cluster size, defined as the weighted standard deviation of the hit positions in $x$, $y$, and $z$. We also compute depth-dependent cluster shapes by evaluating the weighted standard deviations in $\eta$ and $\phi$ as a function of calorimeter layer (hit depth).

4 Physics performance in simulation

MLPF is evaluated in simulation and compared to PF using $\mathrm{t}\overline{\mathrm{t}}$, and QCD samples, and shows realistic particle-level performance, with improved reconstruction efficiency for neutral hadrons while maintaining the same fake rate (Figs. 2).

Jets are clustered using the anti-$k_{\mathrm{T}}$ algorithm with distance parameters $R=0.4$ (AK4), using either PF or MLPF candidates within CMSSW. Jet performance in simulation is evaluated by matching reconstructed jets to generator-level jets within a cone of $\Delta R<0.4$. PF jets use the PUPPI [15, 16] algorithm for PU mitigation prior to clustering. MLPF replaces PUPPI with a learned per-particle PU score for neutral and non-tracker charged particles, while using the standard PUPPI treatment for charged particles within the tracker. Jet and $p_{\mathrm{T}}^{\text{miss}}$ performance are comparable to PF (Figs. 3), despite not being explicitly optimized for during training.

5 Commissioning on data

MLPF was commissioned on a small sample of data collected during the 2024 data-taking period of Run 3. The data are required to pass the dijet high-level trigger, and a dijet offline selection. Reconstructed distributions (e.g., $p_{\mathrm{T}}^{\text{miss}}$, dijet asymmetry) show agreement between PF and MLPF (Fig. 4 left), validating the model’s performance on data collected by CMS.

6 Integration in CMS reconstruction

The model is exported to ONNX and integrated into CMSSW via onnxruntime with FlashAttention kernels for GPU inference. MLPF replaces only the PF reconstruction step, allowing downstream reconstruction modules to run without modification using the particle candidates output by MLPF. Runtime was benchmarked using six parallel CMSSW jobs on a dedicated test machine, each using 8 CPU threads and 1/7th of an NVIDIA A100 80GB GPU. MLPF achieves a $\sim$2× speedup over PF (Fig. 5 right), with runtime of $\sim$40 ms/event on GPU vs. 90 ms/event for PF on CPU. Model outputs are stable across hardware and repeat runs.

7 Summary

We present a machine-learned algorithm for particle-flow (MLPF) reconstruction in the CMS experiment, based on transformers with FlashAttention, demonstrating comparable physics performance, including PU mitigation, while achieving a factor of two reduction in runtime. The model is integrated in CMSSW and validated on simulation and data. Future work includes inference optimization and deployment under HL-LHC Phase-2 conditions.