A novel approach for FPGA-to-server data transmission over an Ethernet-based network using the eXpress Data Path technology

TL;DR

This paper presents a new FPGA-to-server data transmission method over Ethernet using XDP technology, achieving high data throughput with no packet loss during large-scale data transfer for high-energy physics experiments.

Contribution

It introduces an FPGA-to-server data transfer approach leveraging XDP to improve reliability and efficiency in high data rate environments, specifically for CERN's upgraded detectors.

Findings

Transferred 5.2 PB of data without packet loss

Achieved data transmission of 2.92 trillion packets in 168 hours

Utilized XDP to avoid large data buffers in network transmission

Abstract

In the context of the upgrade of the Large Hadron Collider at CERN for high-luminosity operation, the particle detectors have to cope with much higher data rates and therefore need to upgrade their data acquisition systems. This upgrade is taken as an opportunity to exchange the currently used highly customized hardware by commercial solutions. Nevertheless, some part of the data processing still needs to be done within Field Programmable Gate Arrays (FPGA), requiring the transfer of data between the FPGAs and the commercial servers. This paper reports on a study of direct data transmission from FPGAs to servers via a commercial network. Large data buffers as required for reliable data-transmission protocols are avoided by using an emerging technique named eXpress Data Path (XDP). Based on XDP, the transmission of 5.2 PB (i.e. 2.92 * 10^{12} packets) was achieved within 168 h without a single missing packet.