Inner Loop Optimizations in Mapping Single Threaded Programs to Hardware

TL;DR

This paper introduces a dynamic loop-pipelining control mechanism for hardware implementation of single-threaded programs, demonstrating significant performance improvements when combined with loop unrolling on FPGA targets.

Contribution

It presents a novel control-flow mechanism for dynamic loop-pipelining in hardware, enhancing performance of inner loops in single-threaded programs, especially when combined with static loop unrolling.

Findings

Dynamic loop-pipelining improves hardware performance significantly.

Combining loop unrolling with pipelining yields 6X to 20X speedup.

Performance/cost ratio is improved despite hardware overhead.

Abstract

In the context of mapping high-level algorithms to hardware, we consider the basic problem of generating an efficient hardware implementation of a single threaded program, in particular, that of an inner loop. We describe a control-flow mechanism which provides dynamic loop-pipelining capability in hardware, so that multiple iterations of an arbitrary inner loop can be made simultaneously active in the generated hardware, We study the impact of this loop-pipelining scheme in conjunction with source-level loop-unrolling. In particular, we apply this technique to some common loop kernels: regular kernels such as the fast-fourier transform and matrix multiplication, as well as an example of an inner loop whose body has branching. The resulting resulting hardware descriptions are synthesized to an FPGA target, and then characterized for performance and resource utilization. We observe that the use of dynamic loop-pipelining mechanism alone typically results in a significant improvements in the performance of the hardware. If the loop is statically unrolled and if loop-pipelining is applied to the unrolled program, then the performance improvement is still substantial. When dynamic loop pipelining is used in conjunction with static loop unrolling, the improvement in performance ranges from 6X to 20X (in terms of number of clock cycles needed for the computation) across the loop kernels that we have studied. These optimizations do have a hardware overhead, but, in spite of this, we observe that the joint use of these loop optimizations not only improves performance, but also the performance/cost ratio of the resulting hardware.