Decoupled Control Flow and Data Access in RISC-V GPGPUs

TL;DR

This paper proposes architectural modifications to the Vortex RISC-V GPGPU platform, decoupling control flow and data access to significantly improve performance for memory-intensive kernels, fostering future GPGPU research.

Contribution

It introduces a hardware control flow manager and decoupled memory streaming lanes, enabling Vortex to achieve substantial speedups and instruction reduction, advancing open-source GPGPU research.

Findings

8× faster execution for kernels

10× reduction in dynamic instruction count

Performance improved from 0.35 to 1.63 GFLOP/s/mm²

Abstract

Vortex, a newly proposed open-source GPGPU platform based on the RISC-V ISA, offers a valid alternative for GPGPU research over the broadly-used modeling platforms based on commercial GPUs. Similarly to the push originating from the RISC-V movement for CPUs, Vortex can enable a myriad of fresh research directions for GPUs. However, as a young hardware platform, it currently lacks the performance competitiveness of commercial GPUs, which is crucial for widespread adoption. State-of-the-art GPUs, in fact, rely on complex architectural features, still unavailable in Vortex, to hide the micro-code overheads linked to control flow (CF) management and memory orchestration for data access. In particular, these components account for the majority of the dynamic instruction count in regular, memory-intensive kernels, such as linear algebra routines, which form the basis of many applications, including Machine Learning. To address these challenges with simple yet powerful micro-architecture modifications, this paper introduces decoupled CF and data access through 1.) a hardware CF manager to accelerate branching and predication in regular loop execution and 2.) decoupled memory streaming lanes to further hide memory latency with useful computation. The evaluation results for different kernels show 8$\times$ faster execution, 10$\times$ reduction in dynamic instruction count, and overall performance improvement from 0.35 to 1.63 $\mathrm{GFLOP/s/mm^2}$. Thanks to these enhancements, Vortex can become an ideal playground to enable GPGPU research for the next generation of Machine Learning.