JEDI-net: a jet identification algorithm based on interaction networks

TL;DR

JEDI-net is a novel interaction network-based algorithm for jet identification at the LHC, effectively distinguishing heavy particle decays from ordinary jets without complex pre-processing.

Contribution

The paper introduces JEDI-net, a new interaction network architecture that improves jet classification performance with fewer parameters and no special input handling.

Findings

Achieves better classification accuracy than existing methods.

Requires less model complexity and pre-processing.

Demonstrates robustness across different detector geometries.

Abstract

We investigate the performance of a jet identification algorithm based on interaction networks (JEDI-net) to identify all-hadronic decays of high-momentum heavy particles produced at the LHC and distinguish them from ordinary jets originating from the hadronization of quarks and gluons. The jet dynamics are described as a set of one-to-one interactions between the jet constituents. Based on a representation learned from these interactions, the jet is associated to one of the considered categories. Unlike other architectures, the JEDI-net models achieve their performance without special handling of the sparse input jet representation, extensive pre-processing, particle ordering, or specific assumptions regarding the underlying detector geometry. The presented models give better results with less model parameters, offering interesting prospects for LHC applications.

Tables

Table 1. Table 1: Optimal JEDI-net hyperparameter setting for different input data sets, when the summed O ¯ i subscript ¯ 𝑂 𝑖 \overline{O}_{i} quantities are given as input to the ϕ C subscript italic-ϕ 𝐶 \phi_{C} network. The best result, obtained when considering up to 150 particles per jet, is highlighted in bold.

Hyperparameter	Number of jet constituents
	30	50	100	150
$N_n^1$	6	50	30	50
$D_E$	8	12	4	14
$D_O$	6	14	4	10
$f_R$ activation	ReLU	ReLU	SELU	SELU
$f_O$ activation	ELU	ReLU	ReLU	SELU
${\varphi }_C$ activation	ELU	SELU	SELU	SELU
Optimizer	Adam	Adam	Adam	Adam
Optimized loss	0.84	0.58	0.62	0.55

Table 2. Table 2: Optimal JEDI-net hyperparameter setting for different input data sets, when all the O i ​ j subscript 𝑂 𝑖 𝑗 O_{ij} elements are given as input to the ϕ C subscript italic-ϕ 𝐶 \phi_{C} network. The best result, obtained when considering up to 100 particles per jet, is highlighted in bold.

Hyperparameter	Number of jet constituents
	30	50	100	150
$N_n^1$	50	50	30	10
$D_E$	12	12	10	4
$D_O$	6	14	10	14
$f_R$ activation	ReLU	ELU	ELU	SELU
$f_O$ activation	SELU	SELU	ELU	SELU
${\varphi }_C$ activation	SELU	ELU	ELU	SELU
Optimizer	Adam	Adam	Adam	Adam
Optimized loss	0.63	0.57	0.56	0.62

Table 3. Table 3: True positive rates (TPR) for the optimized JEDI-net taggers and the three alternative models (DNN, CNN, and GRU), corresponding to a false positive rate (FPR) of 10% (top) and 1% (bottom). The largest TPR value for each case is highlighted in bold.

Jet category	DNN	GRU	CNN	JEDI-net	JEDI-net
					with $\sum O$
TPR for FPR=$10\%$
gluon	$0.830\pm 0.002$	$0.740\pm 0.014$	$0.700\pm 0.008$	$0.878\pm 0.001$	$0.879\pm 0.001$
light quarks	$0.715\pm 0.002$	$0.746\pm 0.011$	$0.740\pm 0.003$	$0.822\pm 0.001$	$0.818\pm 0.001$
$W$ boson	$0.855\pm 0.001$	$0.812\pm 0.035$	$0.760\pm 0.005$	$0.938\pm 0.001$	$0.927\pm 0.001$
$Z$ boson	$0.833\pm 0.002$	$0.753\pm 0.036$	$0.721\pm 0.006$	$0.910\pm 0.001$	$0.903\pm 0.001$
top quark	$0.917\pm 0.001$	$0.867\pm 0.006$	$0.889\pm 0.001$	$0.930\pm 0.001$	$0.931\pm 0.001$
TPR for FPR=$1\%$
gluon	$0.420\pm 0.002$	$0.273\pm 0.018$	$0.257\pm 0.005$	$0.485\pm 0.001$	$0.482\pm 0.001$
light quarks	$0.178\pm 0.002$	$0.220\pm 0.037$	$0.254\pm 0.007$	$0.302\pm 0.001$	$0.301\pm 0.001$
$W$ boson	$0.656\pm 0.002$	$0.249\pm 0.057$	$0.232\pm 0.006$	$0.704\pm 0.001$	$0.658\pm 0.001$
$Z$ boson	$0.715\pm 0.001$	$0.386\pm 0.060$	$0.291\pm 0.005$	$0.769\pm 0.001$	$0.729\pm 0.001$
top quark	$0.651\pm 0.003$	$0.426\pm 0.020$	$0.504\pm 0.005$	$0.633\pm 0.001$	$0.632\pm 0.001$

Table 4. Table 4: Resource comparison across models. The quoted number of parameters refers only to the trainable parameters for each model. The inference time is measured by applying the model to batches of 1000 events 100 times: the 50% median quantile is quoted as central value and the 10%-90% semi-distance is quoted as the uncertainty. The GPU used is an NVIDIA GTX 1080 with 8 GB memory, mounted on a commercial desktop with an Intel Xeon CPU, operating at a frequency of 2.60GHz. The tests were executed in Python 3.7 with no other concurrent process running on the machine.

Model	Number of	Number of	Inference
	parameters	FLOP	time$/$batch [ms]
DNN	14725	27 k	$1.0\pm 0.2$
CNN	205525	400 k	$57.1\pm 0.5$
GRU	15575	46 k	$23.2\pm 0.6$
JEDI-net	33625	116 M	$121.2\pm 0.4$
JEDI-net	8767	458 M	$402\pm 1$
with $\sum O$

Table 5. Table 5: Optimal CNN hyperparameter setting for different input data sets. The best configuration, used as a benchmark for comparison, is highlighted in bold.

Hyperparameter	Number of jet constituents
	30	50	100	150
$N_{CL}$	3	1	1	3
$n_f$	20	10	30	30
Filter size	$3\times 3$	$\mathrm{𝟑}\times \mathrm{𝟑}$	$3\times 3$	$3\times 3$
Max pooling size	$2\times 2$	$\mathrm{𝟓}\times \mathrm{𝟓}$	$5\times 5$	$2\times 2$
Conv. activation	ReLU	ELU	ELU	ReLU
$N_{DL}$	2	3	3	3
$n_n$	60	50	60	60
Dense activation	SELU	ELU	ELU	ELU
Dropout	0.11	0.1	0.4	0.1
Batch size	200	500	100	50
Optimizer	Adam	Adam	Adam	Adam
Optimized loss	0.88	0.73	0.74	0.74
Optimized accuracy	0.67	0.74	0.74	0.74

Table 6. Table 6: Optimal GRU hyperparameter settings for different input data sets. The best configuration, used as a benchmark for comparison, is highlighted in bold.

Hyperparameter	Number of jet constituents
	30	50	100	150
$n_u$	100	50	200	50
$N_{DL}$	3	1	3	4
$n_n$	70	40	40	100
Dense activation	SELU	SELU	ReLU	ELU
Dropout	0.40	0.10	0.22	0.10
Batch size	500	500	500	500
Optimizer	Adam	Adam	Adam	AdaDelta
Optimized loss	0.78	0.71	0.78	0.85
Optimized accuracy	0.72	0.75	0.73	0.68

Table 7. Table 7: The optimized hyperparameters, number of trainable parameters, and performance metrics of the JEDI-net models on the top tagging data set. Performance metrics are evaluated on the test sample. We quote the area under the ROC curve (AUC), the accuracy, and the background rejection at a signal efficiency of 30%.

Model	JEDI-net	JEDI-net with $\sum O$
Number of constituents	150	150
$N_n^1$	64	256
$D_E$	64	64
$D_O$	16	32
$f_R$ activation	ReLU	SELU
$f_O$ activation	SELU	ReLU
${\varphi }_C$ activation	ReLU	SELU
Optimizer	Adam	Adam
Number of parameters	169906	148962
Accuracy	0.9263	0.9300
AUC	0.9786	0.9807
$1/{ϵ}_B\left({ϵ}_S=30\%\right)$	590.4	774.6

Equations

$$R_S$$

$$R_R$$

$$B$$

$$C$$

Full text

1 Introduction

Jets are collimated cascades of particles produced at particle accelerators. Quarks and gluons originating from hadron collisions, such as the proton-proton collisions at the CERN Large Hadron Collider (LHC), generate a cascade of other particles (mainly other quarks or gluons) that then arrange themselves into hadrons. The stable and unstable hadrons’ decay products are observed by large particle detectors, reconstructed by algorithms that combine the information from different detector components, and then clustered into jets, using physics-motivated sequential recombination algorithms such as those described in Ref. CA ; Catani:1993hr ; Cacciari:2008gp . Jet identification, or tagging, algorithms are designed to identify the nature of the particle that initiated a given cascade, inferring it from the collective features of the particles generated in the cascade.

Traditionally, jet tagging was meant to distinguish three classes of jets: light flavor quarks $\mathrm{q}=\mathrm{u},\mathrm{d},\mathrm{s},\mathrm{c}$, gluons $\mathrm{g}$, or bottom quarks ($\mathrm{b}$). At the LHC, due to the large collision energy, new jet topologies emerge. When heavy particles, e.g. $\mathrm{W}$, $\mathrm{Z}$, or Higgs ($\mathrm{H}$) bosons or the top quark, are produced with large momentum and decay to all-quark final states, the resulting jets are contained in a small solid angle. A single jet emerges from the overlap of two (for bosons) or three (for the top quark) jets, as illustrated in Fig. 1. These jets are characterized by a large invariant mass (computed from the sum of the four-momenta of their constituents) and they differ from ordinary quark and gluon jets, due to their peculiar momentum flow around the jet axis.

Several techniques have been proposed to identify these jets by using physics-motivated quantities, collectively referred to as “jet substructure” variables. A review of the different techniques can be found in Ref. Larkoski:2017jix . As discussed in the review, approaches based on deep learning (DL) have been extensively investigated (see also Sec. 2), processing sets of physics-motivated quantities with dense layers or raw data representations (e.g. jet images or particle feature lists) with more complex architectures (e.g. convolutional or recurrent networks).

In this work, we compare the typical performance of some of these approaches to what is achievable with a novel jet identification algorithm based on an interaction network (JEDI-net). Interaction networks interactionnetwork (INs) were designed to decompose complex systems into distinct objects and relations, and reason about their interactions and dynamics. One of the first uses of INs was to predict the evolution of physical systems under the influence of internal and external forces, for example, to emulate the effect of gravitational interactions in $n$-body systems. The $n$-body system is represented as a set of objects subject to one-on-one interactions. The $n$ bodies are embedded in a graph and these one-on-one interaction functions, expressed as trainable neural networks, are used to predict the post-interaction status of the $n$-body system. We study whether this type of network generalizes to a novel context in high energy physics. In particular, we represent a jet as a set of particles, each of which is represented by its momentum and embedded as a vertex in a fully-connected graph. We use neural networks to learn a representation of each one-on-one particle interaction 111Here, we refer to the abstract message-passing interaction represented by the edges of the graph and not the physical interactions due to quantum chromodynamics, which occur before the jet constituents emerge from the hadronization process. in the jet, which we then use to define jet-related high-level features (HLFs). Based on these features, a classifier associates each jet to one of the five categories shown in Fig. 1.

For comparison, we consider other classifiers based on different architectures: a dense neural network (DNN) MLP receiving a set of jet-substructure quantities, a convolutional neural network (CNN) fukushima ; Cun1990 ; Waibel1990 receiving an image representation of the transverse momentum ($p_{\mathrm{T}}$) flow in the jet 222We use a Cartesian coordinate system with the $z$ axis oriented along the beam axis, the $x$ axis on the horizontal plane, and the $y$ axis oriented upward. The $x$ and $y$ axes define the transverse plane, while the $z$ axis identifies the longitudinal direction. The azimuthal angle $\phi$ is computed from the $x$ axis. The polar angle $\theta$ is used to compute the pseudorapidity $\eta=-\log(\tan(\theta/2))$. We use natural units such that $c=\hbar=1$ and we express energy in units of electronVolt (eV) and its prefix multipliers., and a recurrent neural network (RNN) with gated recurrent units GRU (GRUs), which process a list of particle features. These models can achieve state-of-the-art performance although they require additional ingredients: the DNN model requires processing the constituent particles to pre-compute HLFs, the GRU model assumes an ordering criterion for the input particle feature list, and the CNN model requires representing the jet as a rectangular, regular, pixelated image. Any of these aspects can be handled in a reasonable way (e.g. one can use a jet clustering metric to order the particles), sometimes sacrificing some detector performance (e.g., with coarser image pixels than realistic tracking angular resolution, in the case of many models based on CNN). It is then worth exploring alternative solutions that could reach state-of-the-art performance without making these assumptions. In particular, it is interesting to consider architectures that directly takes as input jet constituents and are invariant for their permutation. This motivated the study of jet taggers based on recursive RecursiveJets , graph networks Egan:2017ojy ; Cheng:2017rdo , and energy flow networks Komiske:2018cqr . In this context, we aim to investigate the potential of INs.

This paper is structured as follows: we provide a list of related works in Sec. 2. In Sec. 3, we describe the utilized data set. The structure of the JEDI-net model is discussed in Sec. 4 together with the alternative architectures considered for comparison. Results are shown in Sec. 5. Sections 6 and 7 discuss what the JEDI-net learns when processing the graph and quantify the amount of resources needed by the tagger, respectively. We conclude with a discussion and outlook for this work in Sec. 8. Appendix A describes the design and optimization of the alternative models.

2 Related work

Jet tagging is one of the most popular LHC-related tasks to which DL solutions have been applied. Several classification algorithms have been studied in the context of jet tagging at the LHC deOliveira:2015xxd ; Guest:2016iqz ; Macaluso:2018tck ; Datta:2017lxt ; Butter:2017cot ; Kasieczka:2017nvn ; Komiske:2016rsd ; Schwartzman:2016jqu using DNNs, CNNs, or physics-inspired architectures. Recurrent and recursive layers have been used to construct jet classifiers starting from a list of reconstructed particle momenta RecursiveJets ; Egan:2017ojy ; Cheng:2017rdo . Recently, these different approaches, applied to the specific case of top quark jet identification, have been compared in Ref. Kasieczka:2019dbj . While many of these studies focus on data analysis, work is underway to apply these algorithms in the early stages of LHC real-time event processing, i.e. the trigger system. For example, Ref. hls4ml focuses on converting these models into firmware for field programmable gate arrays (FPGAs) optimized for low latency (less than 1 $\mu$s). If successful, such a program could allow for a more resource-efficient and effective event selection for future LHC runs.

Graph neural networks have also been considered as jet tagging algorithms graphjettagging ; Qu:2019gqs as a way to circumvent the sparsity of image-based representations of jets. These approaches demonstrate remarkable categorization performance. Motivated by the early results of Ref. graphjettagging , graph networks have been also applied to other high energy physics tasks, such as event topology classification Abdughani:2018wrw ; Choma2018GraphNN , particle tracking in a collider detector GraphTracking , pileup subtraction at the LHC Martinez:2018fwc , and particle reconstruction in irregular calorimeters Qasim:2019otl .

3 Data set description

This study is based on a data set consisting of simulated jets with an energy of $p_{\mathrm{T}}\approx 1$ TeV, originating from light quarks $\mathrm{q}$, gluons $\mathrm{g}$, $\mathrm{W}$ and $\mathrm{Z}$ bosons, and top quarks produced in $\sqrt{s}=13\,\text{TeV}$ proton-proton collisions. The data set was created using the configuration and parametric description of an LHC detector described in Ref. hls4ml ; Coleman:2017fiq , and is available on the Zenodo platform data1 ; data2 ; data3 ; data4 .

Jets are clustered from individual reconstructed particles, using the anti-$k_{\mathrm{T}}$ algorithm Cacciari:2008gp ; Cacciari:2011ma with jet-size parameter $R=0.8$. Three different jet representations are considered:

• •

A list of 16 HLFs, described in Ref. hls4ml , given as input to a DNN. The 16 distributions are shown in Fig. 2 for the five jet classes.

• •

An image representation of the jet, derived by considering a square with pseudorapidity and azimut distances $\Delta\eta=\Delta\phi=2R$, centered along the jet axis. The image is binned into $100\times 100$ pixels. Such a pixel size is comparable to the cell of a typical LHC electromagnetic calorimeter, but much coarser than the typical angular resolution of a tracking device for the $p_{\mathrm{T}}$ values relevant to this task. Each pixel is filled with the scalar sum of the $p_{\mathrm{T}}$ of the particles in that region. These images are obtained by considering the 150 highest-$p_{\mathrm{T}}$ constituents for each jet. This jet representation is used to train a CNN classifier. The average jet images for the five jet classes are shown in Fig. 3. For comparison, a randomly chosen set of images is shown in Fig. 4.

• •

A constituent list for up to 150 particles, in which each particle is represented by 16 features, computed from the particle four-momenta: the three Cartesian coordinates of the momentum ($p_{x}$, $p_{y}$, and $p_{z}$), the absolute energy $E$, $p_{\mathrm{T}}$, the pseudorapidity $\eta$, the azimuthal angle $\phi$, the distance $\Delta R=\sqrt{\Delta\eta^{2}+\Delta\phi^{2}}$ from the jet center, the relative energy $E^{\mathrm{rel}}=E^{\mathrm{particle}}/E^{\mathrm{jet}}$ and relative transverse momentum $p_{\mathrm{T}}^{\mathrm{rel}}=p_{\mathrm{T}}^{\mathrm{particle}}/p_{\mathrm{T}}^{\mathrm{jet}}$ defined as the ratio of the particle quantity and the jet quantity, the relative coordinates $\eta^{\mathrm{rel}}=\eta^{\mathrm{particle}}-\eta^{\mathrm{jet}}$ and $\phi^{\mathrm{rel}}=\phi^{\mathrm{particle}}-\phi^{\mathrm{jet}}$ defined with respect to the jet axis, $\cos{\theta}$ and $\cos{\theta^{\mathrm{rel}}}$ where $\theta^{\mathrm{rel}}=\theta^{\mathrm{particle}}-\theta^{\mathrm{jet}}$ is defined with respect to the jet axis, and the relative $\eta$ and $\phi$ coordinates of the particle after applying a proper Lorentz transformation (rotation) as described in Ref. rotation . Whenever less than 150 particles are reconstructed, the list is filled with zeros. The distributions of these features considering the 150 highest-$p_{\mathrm{T}}$ particles in the jet are shown in Fig. 5 for the five jet categories. This jet representation is used for a RNN with a GRU layer and for JEDI-net.

4 JEDI-net

In this work, we apply an IN interactionnetwork architecture to learn a representation of a given input graph (the set of constituents in a jet) and use it to accomplish a classification task (tagging the jet). One can see the IN architecture as a processing algorithm to learn a new representation of the initial input. This is done replacing a set of input features, describing each individual vertex of the graph, with a set of engineered features, specific of each vertex but whose values depend on the connection between the vertices in the graph.

The starting point consists of building a graph for each input jet. The $N_{O}$ particles in the jet are represented by the vertices of the graph, fully interconnected through directional edges, for a total of $N_{E}=N_{O}\times(N_{O}-1)$ edges. An example is shown in Fig. 6 for the case of a three-vertex graph. The vertices and edges are labeled for practical reasons, but the network architecture ensures that the labeling convention plays no role in creating the new representation.

Once the graph is built, a receiving matrix ($R_{R}$) and a sending matrix ($R_{S}$) are defined. Both matrices have dimensions $N_{O}\times N_{E}$. The element $(R_{R})_{ij}$ is set to 1 when the $i^{\mathrm{th}}$ vertex receives the $j^{\mathrm{th}}$ edge and is 0 otherwise. Similarly, the element $(R_{S})_{ij}$ is set to 1 when the $i^{\mathrm{th}}$ vertex sends the $j^{\mathrm{th}}$ edge and is 0 otherwise. In the case of the graph of Fig. 6, the two matrices take the form:

[TABLE]

The input particle features are represented by an input matrix $I$. Each column of the matrix corresponds to one of the graph vertices, while the rows correspond to the $P$ features used to represent each vertex. In our case, the vertices are the particles inside the jet, each represented by its array of features (i.e., the 16 features shown in Fig. 5). Therefore, the $I$ matrix has dimensions $P\times N_{O}$.

The $I$ matrix is processed by the IN in a series of steps, represented in Fig. 7. The $I$ matrix is multiplied by the $R_{R}$ and $R_{S}$ matrices and the two resulting matrices are then concatenated to form the $B$ matrix, having dimension $2P\times N_{E}$:

[TABLE]

Each column of the $B$ matrix represents an edge, i.e. a particle-to-particle interaction. The $2P$ elements of each column are the features of the sending and receiving vertices for that edge. Using this information, a $D_{E}$-dimensional hidden representation of the interaction edge is created through a trainable function $f_{R}:\mathbb{R}^{2P}\mapsto\mathbb{R}^{D_{E}}$. This gives a matrix $E$ with dimensions $D_{E}\times N_{E}$. The cumulative effects of the interactions received by a given vertex are gathered by summing the $D_{E}$ hidden features over the edges arriving to it. This is done by computing $\overline{E}=ER_{R}^{\top}$ with dimensions $D_{E}\times N_{O}$, which is then appended to the initial input matrix $I$:

[TABLE]

At this stage, each column of the $C$ matrix represents a constituent in the jet, expressed as a $(P+D_{E})$-dimensional feature vector, containing the $P$ input features and the $D_{E}$ hidden features representing the combined effect of the interactions with all the connected particles. A trainable function $f_{O}:\mathbb{R}^{P+D_{E}}\mapsto\mathbb{R}^{D_{O}}$ is used to build a post-interaction representation of each jet constituent. The function $f_{O}$ is applied to each column of $C$ to build the post-interaction matrix $O$ with dimensions $D_{O}\times N_{O}$.

A final classifier $\phi_{C}$ takes as input the elements of the $O$ matrix and returns the probability for that jet to belong to each of the five categories. This is done in two ways: (i) in one case, we define the quantities $\overline{O}_{i}=\sum_{j}O_{ij}$, where $j$ is the index of the vertex in the graph (the particle, in our case), and the $i\in[0,D_{E}]$ index runs across the $D_{E}$ outputs of the $f_{O}$ function. The $\overline{O}$ quantities are used as input to $\phi_{C}:\mathbb{R}^{D_{O}}\mapsto\mathbb{R}^{N}$. This choice allows to preserve the independence of the architecture on the labeling convention adopted to build the $I$, $R_{R}$, and $R_{S}$ matrices, at the cost of losing some discriminating information in the summation. (ii) Alternatively, the $\phi_{C}$ matrix is defined directly from the $D_{O}\times N_{O}$ elements of the $O$ matrix, flattened into a one-dimensional array. The full information from $O$ is preserved, but $\phi_{C}$ assumes an ordering of the $N_{O}$ input objects. In our case, we rank the input particles in descending order by $p_{\mathrm{T}}$.

The trainable functions $f_{O}$, $f_{R}$, and $\phi_{C}$ consist of three DNNs. Each of them has two hidden layers, the first (second) having $N_{n}^{1}$ ($N_{n}^{2}=\lfloor N_{n}^{1}/2\rfloor$) neurons. The model is implemented in PyTorch pytorch and trained using an NVIDIA GTX1080 GPU. The training (validation) data set consists of 630,000 (240,000) examples, while 10,000 events are used for testing purposes.

The architecture of the three trainable functions is determined by minimizing the loss function through a Bayesian optimization, using the GpyOpt library gpyopt2016 , based on Gpy gpy2014 . We consider the following hyperparameters:

• •

The number of output neurons of the $f_{R}$ network, $D_{E}$ (between 4 and 14).

• •

The number of output neurons of the $f_{O}$ network, $D_{O}$ (between 4 and 14).

• •

The number of neurons $N_{n}^{1}$ in the first hidden layer of the $f_{O}$, $f_{R}$, and $\phi_{C}$ network (between 5 and 50).

• •

The activation function for the hidden and output layers of the $f_{R}$ network: ReLU RELU , ELU ELU , or SELU SELU functions.

• •

The activation function for the hidden and output layers of the $f_{O}$ network: ReLU, ELU, or SELU.

• •

The activation function for the hidden layers of the $\phi_{C}$ network: ReLU, ELU, or SELU.

• •

The optimizer algorithm: Adam adam or AdaDelta adadelta .

In addition, the output neurons of the $\phi_{C}$ network are activated by a softmax function. A learning rate of $10^{-4}$ is used. For a given network architecture, the network parameters are optimized by minimizing the categorical cross entropy. The Bayesian optimization is repeated four times. In each case, the input particles are ordered by descending $p_{\mathrm{T}}$ value and the first 30, 50, 100, or 150 particles are considered. The parameter optimization is performed on the training data set, while the loss for the Bayesian optimization is estimated on the validation data set.

Tables 2 and 1 summarize the result of the Bayesian optimization for the JEDI-net architecture with and without the sum over the columns of the $O$ matrix, respectively. The best result of each case, highlighted in bold, is used as a reference for the rest of the paper.

For comparison, three alternative models are trained on the three different representations of the same data set described in Sec. 3: a DNN model taking as input a list of HLFs, a CNN model processing jet images, and a recurrent model applying GRUs on the same input list used for JEDI-net. The three benchmark models are optimized through a Bayesian optimization procedure, as done for the INs. Details of these optimizations and the resulting best models are discussed in Appendix A.

5 Results

Figure 8 shows the receiver operating characteristic (ROC) curves obtained for the optimized JEDI-net tagger in each of the five jet categories, compared to the corresponding curves for the DNN, CNN, and GRU alternative models. The curves are derived by fixing the network architectures to the optimal values based on Table 2 and App. A and performing a $k$-fold cross-validation training, with $k=10$. The solid lines represent the average ROC curve, while the shaded bands quantify the $\pm 1$ RMS dispersion. The area under the curve (AUC) values, reported in the figure, allow for a comparison of the performance of the different taggers.

The algorithm’s tagging performance is quantified computing the true positive rate (TPR) values for two given reference false positive rate (FPR) values (10% and 1%). The comparison of the TPR values gives an assessment of the tagging performance in a realistic use case, typical of an LHC analysis. Tables 3 shows the corresponding FPR values for the optimized JEDI-net taggers, compared to the corresponding values for the benchmark models. The largest TPR value for each class is highlighted in bold. As shown in Fig. 8 and Table 3, the two JEDI-net models outperform the other architectures in almost all cases. The only notable exception is the tight working point of the top-jet tagger, for which the DNN model gives a TPR higher by about $2\%$, while the CNN and GRU models give much worse performance.

The TPR values for the two JEDI-net models are within $1\%$. The only exception is observed for the tight working points of the $\mathrm{W}$ and $\mathrm{Z}$ taggers, for which the model using the $\overline{O}$ sums shows a drop in TPR of $\sim 4\%$. In this respect, the model using summed $\overline{O}$ features is preferable (despite this small TPR loss), given the reduced model complexity (see Section 7) and its independence on the labeling convention for the particles embedded in the graph and for the edges connecting them.

6 What did JEDI-net learn?

In order to characterize the information learned by JEDI-net, we consider the $\overline{O}$ sums across the $N_{O}$ vertices of the graph (see Section 4) and we study their correlations to physics motivated quantities, typically used when exploiting jet substructure in a search. We consider the HLF quantities used for the DNN model and the $N$-subjettiness variables $\tau_{N}^{(\beta)}$ Thaler:2010tr , computed with angular exponent $\beta=1,2$.

Not all the $\overline{O}$ sums exhibit an obvious correlation with the considered quantities, i.e., the network engineers high-level features that encode other information than what is used, for instance, in the DNN model.

Nevertheless, some interesting correlation pattern between the physics motivated quantities and the $\overline{O}_{i}$ sums is observed. The most relevant examples are given in Fig. 9, where the 2D histograms and the corresponding linear correlation coefficient ($\rho$) are shown. The correlation between $\overline{O}_{1}$ and the particle multiplicity in the jet is not completely unexpected. As long as the $O$ quantities aggregated across the graph have the same order of magnitude, the corresponding sum $\overline{O}$ would be proportional to jet-constituent multiplicity.

The strong correlation between the $\overline{O}_{4}$ and $\tau^{(\beta=2)}_{1}$ (with $\rho$ values between 0.69 and 0.97, depending on the jet class) is much less expected. The $\tau_{1}^{\beta}$ quantities assume small values when the jet constituents can be arranged into a single sub-jet inside the jet. Aggregating information from the constituent momenta across the jet, the JEDI-net model based on the $\overline{O}$ quantities learns to build a quantity very close to $\tau^{(\beta=2)}_{1}$. The last two rows of Fig. 9 show two intermediate cases: the correlation between $\overline{O}_{2}$ and $\tau^{(\beta=1)}_{3}$ and between $\overline{O}_{9}$ and $\tau^{(\beta=2)}_{3}$. The two $\overline{O}$ sums considered are correlated to the corresponding substructure quantities, but with smaller (within 0.48 and 0.77) correlation coefficients.

7 Resource comparison

Table 4 shows a comparison of the computational resources needed by the different models discussed in this paper. The best-performing JEDI-net model has more than twice the number of trainable parameters than the DNN and GRU model, but approximately a factor of $6$ less parameters than the CNN model. The JEDI-net model based on the summed $\overline{O}$ features achieves comparable performance with about a factor of $4$ less parameters, less than the DNN and GRU models. While being far from expensive in terms of number of parameters, the JEDI-net models are expensive in terms of the number of floating point operations (FLOP). The simple model based on $\overline{O}$ sums, using as input a sequence of 150 particles, uses 458 MFLOP. The increase is mainly due to the scaling with the number of vertices in the graph. Many of these operations are the $\times 0$ and $\times 1$ products involving the elements of the $R_{R}$ and $R_{S}$ matrices. The cost of these operations could be reduced with an IN implementation optimized for inference, e.g., through an efficient sparse-matrix representation.

In addition, we quote in Table 4 the average inference time on a GPU. The inference time is measured applying the model to 1000 events, as part of a Python application based on TensorFlow tensorflow . To this end, the JEDI-net models, implemented and trained in PyTorch, are exported to ONNX onnx and then loaded as TensorFlow graph. The quoted time includes loading the data, which occurs for the first inference and is different for different event representations, that is smaller for the JEDI-net models than for the CNN models. The GPU used is an NVIDIA GTX 1080 with 8 GB memory, mounted on a commercial desktop with an Intel Xeon CPU, operating at a frequency of 2.60 GHz. The tests were executed in Python 3.7, with no other concurrent process running on the machine. Given the larger number of operations, the GPU inference time for the two IN models is much larger than for the other models.

The current IN algorithm is costly to deploy in the online selection environment of a typical LHC experiment. A dedicated R&D effort is needed to reduce the resource consumption in a realistic environment in order to benefit from the improved accuracy that INs can achieve. For example, one could trade model accuracy for reduced resource needs by applying neural network pruning NIPS1989_250 ; DBLP:journals/corr/HanMD15 , reducing the numerical precision DBLP:journals/corr/abs-1710-09282 ; DBLP:journals/corr/GuptaAGN15 , and limiting the maximum number of particles in each jet representation.

8 Conclusions

This paper presents JEDI-net, a jet tagging algorithm based on interaction networks. Applied to a data set of jets from light-flavor quarks, gluons, vector bosons, and top quarks, this algorithm achieves better performance than models based on dense, convolutional, and recurrent neural networks, trained and optimized with the same procedure on the same data set. As other graph networks, JEDI-net offers several practical advantages that make it particularly suitable for deployment in the data-processing workflows of LHC experiments: it can directly process the list of jet constituent features (e.g. particle four-momenta), it does not assume specific properties of the underlying detector geometry, and it is insensitive to any ordering principle applied to the input jet constituents. For these reasons, the implementation of this and other graph networks is an interesting prospect for future runs of the LHC. On the other hand, the current implementation of this model demands large computational resources and a large inference time, which make the use of these models problematic for real-time selection and calls for a dedicated program to optimize the model deployment on typical L1 and HLT environments.

The quantities engineered by one of the trained IN models exhibit interesting correlation patterns with some of the jet substructure quantities proposed in literature, showing that the model is capable of learning some of the relevant physics in the problem. On the other hand, some of the engineered quantities do not exhibit striking correlation patterns, implying the possibility of a non trivial insight to be gained by studying these quantities.

Acknowledgments

We are grateful to Caltech and the Kavli Foundation for their support of undergraduate student research in cross-cutting areas of machine learning and domain sciences. We would also like to thank the Taylor W. Lawrence Research Fellowship and Mellon Mays Fellowship for supporting E. A. M. and making this research effort possible. This work was conducted at “iBanks,” the AI GPU cluster at Caltech. We acknowledge NVIDIA, SuperMicro and the Kavli Foundation for their support of “iBanks.” This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement no 772369) and is partially supported by the U.S. Department of Energy, Office of High Energy Physics Research under Caltech Contract No. DE-SC0011925. J. M. D. is supported by Fermi Research Alliance, LLC under Contract No. DE-AC02-07CH11359 with the U.S. Department of Energy, Office of Science, Office of High Energy Physics.

Appendix

Appendix A Alternative models

The three benchmark models considered in this work are derived through a Bayesian optimization of their hyperparameters, performed using the GpyOpt library gpyopt2016 , based on Gpy gpy2014 . For each iteration, the training is performed using early stopping to prevent over-fitting and to allow a fair comparison between different configurations. The data set for training (validation) consists of 630,000 (240,000) jets, with 10,000 jets used for testing purposes. The loss for the Bayesian optimization is estimated on the validation data set. The CNN and GRU networks are trained on four different input data sets, obtained considering the first 30, 50, 100, or 150 highest-$p_{\mathrm{T}}$ jet constituents. The DNN model is trained on quantities computed from the full list of particles.

The DNN model consists on a multilayer perceptron, alternating dense layers to dropout layers. The optimal architecture is determined optimizing the following hyperparameters:

• •

Number of dense layers ($N_{DL}$) between 1 and 3.

• •

Number of neurons per dense layer ($n_{n}$): $10,20,\ldots,100$.

• •

Activation functions for the dense layers: ReLU, ELU, or SELU.

• •

Dropout rate: between 0.1 and 0.4.

• •

Batch size: 50, 100, 200, or 500.

• •

Optimization algorithm: Adam, Nadam nadam , or AdaDelta.

The optimization process gives as output an optimal architecture with three hidden layers of 80 neurons each, activated by ELU functions. The best dropout rate is found to be 0.11, when a batch size of 50 and the Adam optimizer are used. This optimized network gives a loss of 0.66 and an accuracy of 0.76.

The CNN model consists of two-dimensional convolutional layers with batch normalization, followed by a set of dense layers. A $2\times 2$ max pooling layer is applied after the fist convolutional layer. The optimal architecture is derived optimizing the following hyperparameters:

• •

Number of convolutional layers $N_{CL}$ between 1 and 3.

• •

Number of convolutional filters $n_{f}$ in each layer (10, 15, 20, 25, or 30).

• •

Convolutional filter size: $3\times 3$, $5\times 5$, $7\times 7$, or $9\times 9$.

• •

Max pooling filter size: $2\times 2$, $3\times 3$, or $5\times 5$.

• •

Activation functions for the convolutional layers (ReLU, ELU, or SELU).

• •

Number of dense layers $N_{DL}$ between 1 and 3.

• •

Number of neurons $n_{n}$ per dense layer: $10,20,\ldots,60$.

• •

Activation functions for the dense layers: ReLU, ELU, or SELU.

• •

Dropout rate: between 0.1 and 0.4.

• •

Batch size: 50, 100, 200, or 500.

• •

Optimization algorithm: Adam, Nadam, or AdaDelta.

The stride of the convolutional filters is fixed to 1 and “same” padding is used. Table 5 shows the optimal sets of hyperparameter values, obtained for the four different data set representations. While the optimal networks are equivalent in performance, we select the network obtained for $\leq 50$ constituents, because it has the smallest number of parameters.

The recurrent model consists of a GRU layer feeding a set of dense layers. The following hyperparameters are considered:

• •

Number of GRU units: 50, 100, 200, 300, 400, or 500.

• •

Activation functions for the GRU layers: ReLU, ELU, or SELU.

• •

Number of dense layers: between 1 and 4.

• •

Number of neurons per dense layer: 10, 20, $\ldots$, 100.

• •

Activation functions for the dense layers: ReLU, ELU, or SELU.

• •

Dropout rate: between 0.1 and 0.4.

• •

Batch size: 50, 100, 200, or 500.

• •

Optimization algorithm: Adam, Nadam, or AdaDelta.

The best hyperparameter values are listed in Table 6. As for the CNN model, the best performance is obtained when the list of input particles is truncated at 50 elements.

Appendix B Performance on public top tagging data set

In this appendix, we retrain and evaluate the performance of JEDI-net on a public top tagging data set Butter:2017cot ; Kasieczka:2019dbj used to benchmark many neural networks architectures for the task of differentiating top quark jets from light quark jets. To select the hyperparameters of the model (with and without the sum over particles), we performed a Bayesian optimization. We scan $N_{n}^{1}$ from $16$ to $256$, $D_{E}$ from $4$ to $64$, $D_{O}$ from $4$ to $64$, ReLU, ELU, or SELU activation functions for $f_{R}$, $f_{O}$, and $\phi_{C}$, and either the Adam or Adadelta optimizers with an initial learning rate of $10^{-3}$. We report three metrics for the performance of the network on the top tagging data set: model accuracy, area under the ROC curve (AUC), and background rejection power at a fixed signal efficiency of 30%, $1/\epsilon_{B}(\epsilon_{S}=30\%)$. In Table 7, the accuracy, AUC, and $1/\epsilon_{B}(\epsilon_{S}=30\%)$ values are listed for each model considered. The performance of JEDI-net compared to other models developed for this data set is approaching state-of-the-art Kasieczka:2019dbj .