FPGA-based Multi-Chip Module for High-Performance Computing

TL;DR

This paper presents the successful design, fabrication, and testing of FPGA-based multi-chip modules tailored for Exascale high-performance computing, demonstrating high-speed interconnect stability and robust manufacturing quality.

Contribution

It introduces a novel FPGA-based multi-chip module architecture specifically designed for Exascale computing, with successful fabrication and comprehensive testing results.

Findings

High-speed intra-board links stable at 10 Gbps

Successful fabrication with no manufacturing defects

Effective testing of DDR memory and FPGA links

Abstract

Current integration, architectural design and manufacturing technologies are not suited for the computing density and power efficiency requested by Exascale computing. New approaches in hardware architecture are thus needed to overcome the technological barriers preventing the transition to the Exascale era. In that scope, we report successful fabrication of first ExaNoDe's MCM prototypes dedicated to Exascale computing applications. Each MCM was composed of 2 Xilinx Zynq Ultrascale+ MPSoC, assembled on advanced 68.5 mm x 55 mm laminate substrates specifically designed and fabricated for the project. Acoustic microscopy, x-ray, cross-section and Thermo-Moire investigations revealed no voids, shorts, delamination, cracks or warpage issues. Two MCMs were mounted on a daughter board by FORTH for testing purposes. The DDR memories on the 4 SODIMMs of the daughter board were successfully tested by running extensive Xilinx memory tests with clock frequencies of 1866 MHz and 2133 MHz. All 4 FPGAs were programmed with the Xilinx integrated bit error ratio test (IBERT) tailored for this board for links testing. All intra-board high-speed links between all FPGAs were stable at 10 Gbps, even under the more demanding 31-bit PRBS (Pseudorandom Binary Sequence) tests.