Inter-thread Communication in Multithreaded, Reconfigurable Coarse-grain Arrays

TL;DR

This paper proposes a novel direct inter-thread communication method in multithreaded CGRAs, significantly improving performance and power efficiency over traditional GPGPU memory-based communication.

Contribution

It introduces a new communication model, hardware primitives, and system extensions enabling direct thread-to-thread data exchange in CGRAs.

Findings

Average speedup of 4.5x over GPGPU

Power reduction of 7x on average

Elimination of barriers and scratchpad memory

Abstract

Traditional von Neumann GPGPUs only allow threads to communicate through memory on a group-to-group basis. In this model, a group of producer threads writes intermediate values to memory, which are read by a group of consumer threads after a barrier synchronization. To alleviate the memory bandwidth imposed by this method of communication, GPGPUs provide a small scratchpad memory that prevents intermediate values from overloading DRAM bandwidth. In this paper we introduce direct inter-thread communications for massively multithreaded CGRAs, where intermediate values are communicated directly through the compute fabric on a point-to-point basis. This method avoids the need to write values to memory, eliminates the need for a dedicated scratchpad, and avoids workgroup-global barriers. The paper introduces the programming model (CUDA) and execution model extensions, as well as the hardware primitives that facilitate the communication. Our simulations of Rodinia benchmarks running on the new system show that direct inter-thread communication provides an average speedup of 4.5x (13.5x max) and reduces system power by an average of 7x (33x max), when compared to an equivalent Nvidia GPGPU.