Supporting High-Performance and High-Throughput Computing for Experimental Science

TL;DR

This paper advocates for integrating high-performance and high-throughput computing infrastructures to better support large-scale scientific experiments, illustrated through diverse case studies across scientific domains.

Contribution

It introduces the concept of unified infrastructure for high-performance and high-throughput computing in experimental science, supported by multiple case studies.

Findings

Unified infrastructure enhances scientific discovery capabilities.

Case studies demonstrate successful integration across domains.

Identifies common requirements for such integrated systems.

Abstract

The advent of experimental science facilities-instruments and observatories, such as the Large Hadron Collider, the Laser Interferometer Gravitational Wave Observatory, and the upcoming Large Synoptic Survey Telescope-has brought about challenging, large-scale computational and data processing requirements. Traditionally, the computing infrastructure to support these facility's requirements were organized into separate infrastructure that supported their high-throughput needs and those that supported their high-performance computing needs. We argue that to enable and accelerate scientific discovery at the scale and sophistication that is now needed, this separation between high-performance computing and high-throughput computing must be bridged and an integrated, unified infrastructure provided. In this paper, we discuss several case studies where such infrastructure has been implemented. These case studies span different science domains, software systems, and application requirements as well as levels of sustainability. A further aim of this paper is to provide a basis to determine the common characteristics and requirements of such infrastructure, as well as to begin a discussion of how best to support the computing requirements of existing and future experimental science facilities.