HEPCloud, an Elastic Hybrid HEP Facility using an Intelligent Decision Support System
Parag Mhashilkar, Mine Altunay, Eileen Berman, David Dagenhart, Stuart, Fuess, Burt Holzman, James Kowalkowski, Dmitry Litvintsev, Qiming Lu,, Alexander Moibenko, Marc Paterno, Panagiotis Spentzouris, Steven Timm,, Anthony Tiradani, Eric Vaandering (Fermilab), John Hover

TL;DR
HEPCloud is an elastic, hybrid computing facility for high energy physics that uses an intelligent decision support system to automatically provision resources across multiple cloud and HPC providers, optimizing costs and meeting experimental demands.
Contribution
The paper introduces a modular intelligent decision support system that automates resource provisioning across diverse computing platforms for high energy physics experiments.
Findings
Successful automated expansion of HEPCloud at Fermilab and Brookhaven.
Demonstrated cost-effective resource management across multiple providers.
Validated the system's ability to meet peak computational demands.
Abstract
HEPCloud is rapidly becoming the primary system for provisioning compute resources for all Fermilab-affiliated experiments. In order to reliably meet the peak demands of the next generation of High Energy Physics experiments, Fermilab must plan to elastically expand its computational capabilities to cover the forecasted need. Commercial cloud and allocation-based High Performance Computing (HPC) resources both have explicit and implicit costs that must be considered when deciding when to provision these resources, and at which scale. In order to support such provisioning in a manner consistent with organizational business rules and budget constraints, we have developed a modular intelligent decision support system (IDSS) to aid in the automatic provisioning of resources spanning multiple cloud providers, multiple HPC centers, and grid computing federations. In this paper, we discuss the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReservoir Engineering and Simulation Methods
