Building a Distributed Computing System for LDMX: Challenges of creating and operating a lightweight e-infrastructure for small-to-medium size accelerator experiments

TL;DR

This paper discusses the development and testing of a lightweight distributed computing system for the LDMX experiment, demonstrating that leveraging existing technologies can simplify infrastructure setup for small-to-medium scale scientific collaborations.

Contribution

It introduces a scalable, low-overhead distributed computing solution tailored for small-scale accelerator experiments, emphasizing integration of existing tools to reduce development effort.

Findings

Successful deployment of a pilot system for large-scale simulations

Significant reduction in setup and operational effort

Enhanced scalability and reproducibility of computing resources

Abstract

Particle physics experiments rely extensively on computing and data services, making e-infrastructure an integral part of the research collaboration. Constructing and operating distributed computing can however be challenging for a smaller-scale collaboration. The Light Dark Matter eXperiment (LDMX) is a planned small-scale accelerator-based experiment to search for dark matter in the sub-GeV mass region. Finalizing the design of the detector relies on Monte-Carlo simulation of expected physics processes. A distributed computing pilot project was proposed to better utilize available resources at the collaborating institutes, and to improve scalability and reproducibility. This paper outlines the chosen lightweight distributed solution, presenting requirements, the component integration steps, and the experiences using a pilot system for tests with large-scale simulations. The system leverages existing technologies wherever possible, minimizing the need for software development, and deploys only non-intrusive components at the participating sites. The pilot proved that integrating existing components can dramatically reduce the effort needed to build and operate a distributed e-infrastructure, making it attainable even for smaller research collaborations.