Evolutionary algorithms for hyperparameter optimization in machine learning for application in high energy physics

TL;DR

This paper investigates the use of evolutionary algorithms, specifically particle swarm optimization and genetic algorithms, to automate hyperparameter tuning in machine learning applications for high energy physics data analysis.

Contribution

It introduces the application of PSO and GA for hyperparameter optimization in high energy physics ML tasks, comparing their effectiveness to other methods.

Findings

PSO and GA effectively optimize hyperparameters in high energy physics ML models.

Evolutionary algorithms outperform manual tuning in certain scenarios.

Automated hyperparameter tuning improves ML model performance and reduces manual effort.

Abstract

The analysis of vast amounts of data constitutes a major challenge in modern high energy physics experiments. Machine learning (ML) methods, typically trained on simulated data, are often employed to facilitate this task. Several choices need to be made by the user when training the ML algorithm. In addition to deciding which ML algorithm to use and choosing suitable observables as inputs, users typically need to choose among a plethora of algorithm-specific parameters. We refer to parameters that need to be chosen by the user as hyperparameters. These are to be distinguished from parameters that the ML algorithm learns autonomously during the training, without intervention by the user. The choice of hyperparameters is conventionally done manually by the user and often has a significant impact on the performance of the ML algorithm. In this paper, we explore two evolutionary algorithms: particle swarm optimization (PSO) and genetic algorithm (GA), for the purposes of performing the choice of optimal hyperparameter values in an autonomous manner. Both of these algorithms will be tested on different datasets and compared to alternative methods.