Automatic Nested Loop Acceleration on FPGAs Using Soft CGRA Overlay

TL;DR

This paper presents an automatic framework for customizing FPGA-based soft CGRA overlays to accelerate nested loops, significantly improving performance and productivity compared to non-customized solutions.

Contribution

It introduces an automated method for optimizing overlay architecture and compilation parameters for nested loop acceleration on FPGAs, enhancing performance with minimal additional runtime.

Findings

Up to 5x speedup over non-customized accelerators.

Up to 10x speedup compared to host-only software execution.

Customization process takes 10-20 minutes.

Abstract

Offloading compute intensive nested loops to execute on FPGA accelerators have been demonstrated by numerous researchers as an effective performance enhancement technique across numerous application domains. To construct such accelerators with high design productivity, researchers have increasingly turned to the use of overlay architectures as an intermediate generation target built on top of off-the-shelf FPGAs. However, achieving the desired performance-overhead trade-off remains a major productivity challenge as complex application-specific customizations over a large design space covering multiple architectural parameters are needed. In this work, an automatic nested loop acceleration framework utilizing a regular soft coarse-grained reconfigurable array (SCGRA) overlay is presented. Given high-level resource constraints, the framework automatically customizes the overlay architectural design parameters, high-level compilation options as well as communication between the accelerator and the host processor for optimized performance specifically to the given application. In our experiments, at a cost of 10 to 20 minutes additional tools run time, the proposed customization process resulted in up to 5 times additional speedup over a baseline accelerator generated by the same framework without customization. Overall, when compared to the equivalent software running on the host ARM processor alone on the Zedboard, the resulting accelerators achieved up to 10 times speedup.