Using ATLAS@Home to exploit extra CPU from busy grid sites

TL;DR

This paper demonstrates how ATLAS@Home can effectively utilize idle CPU resources on grid sites, increasing overall efficiency without impacting primary workloads or job success rates.

Contribution

It introduces a backfilling approach using ATLAS@Home to enhance CPU utilization on grid sites, maintaining performance and reliability.

Findings

CPU utilization increased by 15% to 42%.

Overall CPU utilization remained over 90%.

Backfilling did not affect grid job failure rates.

Abstract

Grid computing typically provides most of the data processing resources for large High Energy Physics experiments. However typical grid sites are not fully utilized by regular workloads. In order to increase the CPU utilization of these grid sites, the ATLAS@Home volunteer computing framework can be used as a backfilling mechanism. Results show an extra 15% to 42% of CPU cycles can be exploited by backfilling grid sites running regular workloads while the overall CPU utilization can remain over 90%. Backfilling has no impact on the failure rate of the grid jobs, and the impact on the CPU efficiency of grid jobs varies from 1% to 11% depending on the configuration of the site. In addition the throughput of backfill jobs in terms of CPU time per simulated event is the same as for resources dedicated to ATLAS@Home. This approach is sufficiently generic that it can easily be extended to other clusters.