Compiler Support for Speculation in Decoupled Access/Execute Architectures

TL;DR

This paper introduces compiler techniques for speculative memory requests in decoupled access/execute architectures, enabling better decoupling despite control dependencies, applicable to various hardware platforms.

Contribution

It presents a novel compiler-based speculation method for DAE architectures that maintains decoupling with control dependencies, proven to preserve sequential consistency.

Findings

Enables DAE in more codes by handling control dependencies.

Works on arbitrary reducible control flow.

Applicable to CPU, GPU, CGRA, and accelerator architectures.

Abstract

Irregular codes are bottlenecked by memory and communication latency. Decoupled access/execute (DAE) is a common technique to tackle this problem. It relies on the compiler to separate memory address generation from the rest of the program, however, such a separation is not always possible due to control and data dependencies between the access and execute slices, resulting in a loss of decoupling. In this paper, we present compiler support for speculation in DAE architectures that preserves decoupling in the face of control dependencies. We speculate memory requests in the access slice and poison mis-speculations in the execute slice without the need for replays or synchronization. Our transformation works on arbitrary, reducible control flow and is proven to preserve sequential consistency. We show that our approach applies to a wide range of architectural work on CPU/GPU prefetchers, CGRAs, and accelerators, enabling DAE on a wider range of codes than before.