APEnet+: high bandwidth 3D torus direct network for petaflops scale commodity clusters

TL;DR

The paper presents APEnet+, a high-bandwidth 3D torus network architecture designed for scalable petaflops clusters, featuring advanced interconnect hardware, RDMA support, and GPU acceleration.

Contribution

It introduces the APEnet+ PCIe interconnect adapter and demonstrates its capability to build low latency, high bandwidth, scalable PC clusters for petaflops-scale computing.

Findings

Achieved high data transmission rates in testbench

Demonstrated low latency in network communication

Validated hardware support for RDMA and GPU acceleration

Abstract

We describe herein the APElink+ board, a PCIe interconnect adapter featuring the latest advances in wire speed and interface technology plus hardware support for a RDMA programming model and experimental acceleration of GPU networking; this design allows us to build a low latency, high bandwidth PC cluster, the APEnet+ network, the new generation of our cost-effective, tens-of-thousands-scalable cluster network architecture. Some test results and characterization of data transmission of a complete testbench, based on a commercial development card mounting an Altera FPGA, are provided.