Systematic benchmarking of HTTPS third party copy on 100Gbps links using XRootD

TL;DR

This paper systematically benchmarks the performance of HTTPS third-party copy protocols, specifically XRootD-HTTPS and GridFTP, over 100Gbps links using a Kubernetes-based testing framework, aiding future network protocol evaluations.

Contribution

It introduces a reproducible benchmarking methodology for TPC protocols on high-speed links, focusing on XRootD-HTTPS and GridFTP performance evaluation.

Findings

XRootD-HTTPS achieves competitive transfer speeds.

Benchmarking methodology ensures reproducibility and repeatability.

Framework can be used for testing other TPC protocols.

Abstract

The High Luminosity Large Hadron Collider provides a data challenge. The amount of data recorded from the experiments and transported to hundreds of sites will see a thirty fold increase in annual data volume. A systematic approach to contrast the performance of different Third Party Copy(TPC) transfer protocols arises. Two contenders, XRootD-HTTPS and the GridFTP are evaluated in their performance for transferring files from one server to an-other over 100Gbps interfaces. The benchmarking is done by scheduling pods on the Pacific Research Platform Kubernetes cluster to ensure reproducible and repeatable results. This opens a future pathway for network testing of any TPC transfer protocol.