Searches for the BSM scenarios at the LHC using decision tree based machine learning algorithms: A comparative study and review of Random Forest, Adaboost, XGboost and LightGBM frameworks

TL;DR

This paper reviews and compares decision tree machine learning algorithms like Random Forest, AdaBoost, XGBoost, and LightGBM in the context of high energy physics, demonstrating their effectiveness in SUSY searches at the LHC.

Contribution

It provides a comprehensive review and comparison of decision tree algorithms in high energy physics, including a case study showing improved search sensitivity for SUSY at the LHC.

Findings

Decision tree algorithms improve search sensitivity over traditional methods.

Hyperparameter optimization enhances model performance.

Feature importance analysis aids in understanding model decisions.

Abstract

Machine learning algorithms are now being extensively used in our daily lives, spanning across diverse industries as well as academia. In the field of high energy physics (HEP), the most common and challenging task is separating a rare signal from a much larger background. The boosted decision tree (BDT) algorithm has been a cornerstone of the high energy physics for analyzing event triggering, particle identification, jet tagging, object reconstruction, event classification, and other related tasks for quite some time. This article presents a comprehensive overview of research conducted by both HEP experimental and phenomenological groups that utilize decision tree algorithms in the context of the Standard Model and Supersymmetry (SUSY). We also summarize the basic concept of machine learning and decision tree algorithm along with the working principle of \texttt{Random Forest}, \texttt{AdaBoost} and two gradient boosting frameworks, such as \texttt{XGBoost}, and \texttt{LightGBM}. Using a case study of electroweakino productions at the high luminosity LHC, we demonstrate how these algorithms lead to improvement in the search sensitivity compared to traditional cut-based methods in both compressed and non-compressed R-parity conserving SUSY scenarios. The effect of different hyperparameters and their optimization, feature importance study using SHapley values are also discussed in detail.