Improving the Efficiency of OpenCL Kernels through Pipes

TL;DR

This paper explores using pipes in OpenCL kernels to enhance memory bandwidth utilization on FPGAs, leading to notable performance gains especially for irregular memory access patterns.

Contribution

It introduces a method leveraging pipes to separate memory access from computation, improving bandwidth utilization and performance on FPGA-based OpenCL kernels.

Findings

Significant performance improvements for certain kernels

Enhanced memory bandwidth utilization with pipes

Effective for irregular memory access patterns

Abstract

In an effort to lower the barrier to the adoption of FPGAs by a broader community, today major FPGA vendors offer compiler toolchains for OpenCL code. While using these toolchain allows porting existing code to FPGAs, ensuring performance portability across devices (i.e., CPUs, GPUs and FPGAs) is not a trivial task. This is in part due to the different hardware characteristics of these devices, including the nature of the hardware parallelism and the memory bandwidth they offer. In particular, global memory accesses are known to be one of the main performance bottlenecks for OpenCL kernels deployed on FPGA. In this paper, we investigate the use of pipes to improve memory bandwidth utilization and performance of OpenCL kernels running on FPGA. This is done by separating the global memory accesses from the computation, enabling better use of the load units required to access global memory. We perform experiments on a set of broadly used benchmark applications with various compute and memory access patterns. Our experiments, conducted on an Intel Arria GX board, show that the proposed method is effective in improving the memory bandwidth utilization of most kernels, particularly those exhibiting irregular memory access patterns. This, in turn, leads to performance improvements, in some cases significant.