Snowmass Computational Frontier: Topical Group Report on Experimental Algorithm Parallelization

TL;DR

This report discusses the challenges and solutions for developing and parallelizing experimental algorithms in high-energy physics, emphasizing heterogeneous computing, software sharing, workforce development, and cross-disciplinary collaboration for future experiments.

Contribution

It provides a comprehensive overview of strategies and infrastructure needs for experimental algorithm development and parallelization over the next 10-15 years.

Findings

Need for portable, shared software tools across experiments.

Importance of workforce development in algorithm research.

Challenges posed by heterogeneous computing architectures.

Abstract

The substantial increase in data volume and complexity expected from future experiments will require significant investment to prepare experimental algorithms. These algorithms include physics object reconstruction, calibrations, and processing of observational data. In addition, the changing computing architecture landscape, which will be primarily composed of heterogeneous resources, will continue to pose major challenges with regard to algorithmic migration. Portable tools need to be developed that can be shared among the frontiers (e.g., for code execution on different platforms) and opportunities, such as forums or cross-experimental working groups, need to be provided where experiences and lessons learned can be shared between experiments and frontiers. At the same time, individual experiments also need to invest considerable resources to develop algorithms unique to their needs (e.g., for facilities dedicated to the experiment), and ensure that their specific algorithms will be able to efficiently exploit external heterogeneous computing facilities. Common software tools represent a cost-effective solution, providing ready-to-use software solutions as well as a platform for R\&D work. These are particularly important for small experiments which typically do not have dedicated resources needed to face the challenges imposed by the evolving computing technologies. Workforce development is a key concern across frontiers and experiments, and additional support is needed to provide career opportunities for researchers working in the field of experimental algorithm development. Finally, cross-discipline collaborations going beyond high-energy physics are a key ingredient to address the challenges ahead and more support for such collaborations needs to be created. This report targets future experiments, observations and experimental algorithm development for the next 10-15 years.