IceCube experience using XRootD-based Origins with GPU workflows in PNRP

TL;DR

This paper discusses IceCube's use of XRootD-based Origins with GPU workflows on the PNRP platform to support large-scale simulation updates, highlighting experiences, challenges, and improvements in systematic effect modeling.

Contribution

It presents the first implementation of XRootD-based OSDF Origins for IceCube, integrating GPU workflows on PNRP to enhance simulation processing capabilities.

Findings

Successful deployment of XRootD-based storage for IceCube

Enhanced GPU workflow integration for simulation updates

Identified challenges in storage and workflow management

Abstract

The IceCube Neutrino Observatory is a cubic kilometer neutrino telescope located at the geographic South Pole. Understanding detector systematic effects is a continuous process. This requires the Monte Carlo simulation to be updated periodically to quantify potential changes and improvements in science results with more detailed modeling of the systematic effects. IceCube's largest systematic effect comes from the optical properties of the ice the detector is embedded in. Over the last few years there have been considerable improvements in the understanding of the ice, which require a significant processing campaign to update the simulation. IceCube normally stores the results in a central storage system at the University of Wisconsin-Madison, but it ran out of disk space in 2022. The Prototype National Research Platform (PNRP) project thus offered to provide both GPU compute and storage capacity to IceCube in support of this activity. The storage access was provided via XRootD-based OSDF Origins, a first for IceCube computing. We report on the overall experience using PNRP resources, with both successes and pain points.