DPUV4E: High-Throughput DPU Architecture Design for CNN on Versal ACAP

TL;DR

This paper introduces DPUV4E, a high-throughput DPU architecture for CNN acceleration on AMD's Versal ACAP, achieving significant improvements in efficiency, resource utilization, and inference throughput.

Contribution

The paper presents a novel DPU design tailored for Versal ACAP, supporting diverse configurations and computation units, and extends functionality to non-convolutional operations, enhancing performance and resource efficiency.

Findings

8.6× TOPS/W compared to traditional FPGA DPUs

95.8% reduction in DSP usage

Up to 2.2× throughput improvement for depth-wise convolution models

Abstract

Convolutional Neural Networks (CNNs) remain prevalent in computer vision applications, and FPGAs, known for their flexibility and energy efficiency, have become essential components in heterogeneous acceleration systems. However, traditional FPGAs face challenges in balancing performance and versatility due to limited on-chip resources. AMD's Versal ACAP architecture, tailored for AI applications, incorporates AI Engines (AIEs) to deliver high computational power. Nevertheless, the platform suffers from insufficient memory bandwidth, hindering the full utilization of the AIEs' theoretical performance. In this paper, we present DPUV4E for the Versal architecture, providing configurations ranging from 2PE ($32.6$ TOPS) to 8PE ($131.0$ TOPS). We design two computation units, Conv PE and DWC PE, to support different computational patterns. Each computation unit's data flow efficiently utilizes the data reuse opportunities to mitigate bandwidth bottlenecks. Additionally, we extend the functionality of each PE to utilize AIEs for non-convolutional operations, reducing resource overhead. Experiments on over 50 models show that compared to previous designs, our design provides $8.6\times$ the TOPS/W of traditional FPGA-based DPU designs, while reducing DSP usage by $95.8\%$, LUT usage by $44.7\%$, and latency to $68.5\%$ under single-batch conditions. For end-to-end inference, our design improving throughput by up to $2.2\times$ for depth-wise convolution models and up to $1.3\times$ for standard models.