Third-party transfers in WLCG using HTTP

TL;DR

This paper explores the HTTP-TPC protocol as a modern, secure alternative to FTP for data transfers in the WLCG, highlighting its support, interoperability, and potential for integrating advanced authorization methods.

Contribution

It introduces the HTTP-TPC protocol, discusses its implementation status, interoperability, and demonstrates its capability to support new authorization technologies in WLCG.

Findings

HTTP-TPC supports interoperability within WLCG.

Token-based authorization enables integration of OAuth, OpenID Connect, Macaroons, SciTokens.

HTTP-TPC offers a viable alternative to FTP for large-scale data transfers.

Abstract

Since its earliest days, the Worldwide LHC Computational Grid (WLCG) has relied on GridFTP to transfer data between sites. The announcement that Globus is dropping support of its open source Globus Toolkit (GT), which forms the basis for several FTP client and servers, has created an opportunity to reevaluate the use of FTP. HTTP-TPC, an extension to HTTP compatible with WebDAV, has arisen as a strong contender for an alternative approach. In this paper, we describe the HTTP-TPC protocol itself, along with the current status of its support in different implementations, and the interoperability testing done within the WLCG DOMA working group's TPC activity. This protocol also provides the first real use-case for token-based authorisation for this community. We will demonstrate the benefits of such authorisation by showing how it allows HTTP-TPC to support new technologies (such as OAuth, OpenID Connect, Macaroons and SciTokens) without changing the protocol. We will also discuss the next steps for HTTP-TPC and the plans to use the protocol for WLCG transfers.