Architectural improvements and technological enhancements for the APEnet+ interconnect system

TL;DR

This paper presents the latest APEnet+ v5 interconnect system, a PCIe Gen3 FPGA-based network interface card designed for low-latency, high-bandwidth GPU communication in 3D torus networks, with improved architecture and technology.

Contribution

Introduction of APEnet v5, a new generation NIC with enhanced bandwidth, low latency, and integration of state-of-the-art transceivers and PCIe Gen3 compliance.

Findings

Achieved high bandwidth performance

Demonstrated low bit error rate (BER)

Validated PCIe Gen3 compatibility

Abstract

The APEnet+ board delivers a point-to-point, low-latency, 3D torus network interface card. In this paper we describe the latest generation of APEnet NIC, APEnet v5, integrated in a PCIe Gen3 board based on a state-of-the-art, 28 nm Altera Stratix V FPGA. The NIC features a network architecture designed following the Remote DMA paradigm and tailored to tightly bind the computing power of modern GPUs to the communication fabric. For the APEnet v5 board we show characterizing figures as achieved bandwidth and BER obtained by exploiting new high performance ALTERA transceivers and PCIe Gen3 compliancy.