NetDAM: Network Direct Attached Memory with Programmable In-Memory Computing ISA

TL;DR

NetDAM introduces a programmable network-attached memory architecture with in-memory computing capabilities, enabling high-performance data transfer and processing for large-scale distributed applications, demonstrated through FPGA prototype and MPI-Allreduce case.

Contribution

It presents a novel FPGA-based architecture that combines network-attached memory with programmable in-memory computing for efficient data-intensive operations.

Findings

FPGA prototype demonstrates high performance in MPI-Allreduce

NetDAM offers a flexible alternative to RDMA for data transfer

Effective integration of memory pooling and in-network computing

Abstract

Data-intensive applications like distributed AI-training may require multi-terabytes memory capacity with multi-terabits bandwidth. We directly attach the memory to the ethernet controller with some programable logic to design an efficient hardware "template" for Memory pooling and in-memory / in-network computing. We built an FPGA prototype of the NetDAM, andwe demonstrate MPI-Allreduce communication case, the NetDAM can be used as a software and hardware friendly programmable architeture with high performance alternative for RDMA.