Efficient and Robust Jet Tagging at the LHC with Knowledge Distillation

TL;DR

This paper introduces a knowledge distillation approach to develop efficient, robust jet tagging models for the LHC, achieving high performance with low computational cost and improved robustness by transferring Lorentz symmetry bias.

Contribution

It demonstrates how knowledge distillation can produce small, efficient models that retain high accuracy and robustness, incorporating Lorentz symmetry inductive bias from large teacher models.

Findings

Student models outperform baseline small models in jet classification.

Knowledge distillation improves robustness against Lorentz boosts.

Incorporating Lorentz symmetry enhances model generalization.

Abstract

The challenging environment of real-time data processing systems at the Large Hadron Collider (LHC) strictly limits the computational complexity of algorithms that can be deployed. For deep learning models, this implies that only models with low computational complexity that have weak inductive bias are feasible. To address this issue, we utilize knowledge distillation to leverage both the performance of large models and the reduced computational complexity of small ones. In this paper, we present an implementation of knowledge distillation, demonstrating an overall boost in the student models' performance for the task of classifying jets at the LHC. Furthermore, by using a teacher model with a strong inductive bias of Lorentz symmetry, we show that we can induce the same inductive bias in the student model which leads to better robustness against arbitrary Lorentz boost.