FPGA Processor In Memory Architectures (PIMs): Overlay or Overhaul ?

TL;DR

This paper compares FPGA-based PIM architectures and overlays, demonstrating that a PIM overlay can achieve high throughput and efficiency, and can enhance custom PIM designs.

Contribution

It provides a comparative analysis of custom FPGA PIM architectures and a novel PIM overlay, introducing PiCaSO and showing how overlay features improve custom designs.

Findings

PiCaSO achieves 80% of peak throughput of custom designs

PiCaSO has 2.56x shorter latency than custom architectures

Overlay features improve custom PIM throughput by 18%, latency by 19.5%, and efficiency by 6.2%

Abstract

The dominance of machine learning and the ending of Moore's law have renewed interests in Processor in Memory (PIM) architectures. This interest has produced several recent proposals to modify an FPGA's BRAM architecture to form a next-generation PIM reconfigurable fabric. PIM architectures can also be realized within today's FPGAs as overlays without the need to modify the underlying FPGA architecture. To date, there has been no study to understand the comparative advantages of the two approaches. In this paper, we present a study that explores the comparative advantages between two proposed custom architectures and a PIM overlay running on a commodity FPGA. We created PiCaSO, a Processor in/near Memory Scalable and Fast Overlay architecture as a representative PIM overlay. The results of this study show that the PiCaSO overlay achieves up to 80% of the peak throughput of the custom designs with 2.56x shorter latency and 25% - 43% better BRAM memory utilization efficiency. We then show how several key features of the PiCaSO overlay can be integrated into the custom PIM designs to further improve their throughput by 18%, latency by 19.5%, and memory efficiency by 6.2%.