A Case for Fine-grain Coherence Specialization in Heterogeneous Systems

TL;DR

This paper advocates for fine-grain coherence specialization in heterogeneous systems, demonstrating significant performance and traffic reductions through optimized memory request handling on a flexible coherence interface.

Contribution

It introduces a novel architecture that enables independent optimization of coherence requests, enhancing performance and efficiency in heterogeneous accelerators.

Findings

Reduced execution time by up to 61%

Network traffic decreased by up to 99%

Achieved minimal complexity increase in coherence protocol

Abstract

Hardware specialization is becoming a key enabler of energyefficient performance. Future systems will be increasingly heterogeneous, integrating multiple specialized and programmable accelerators, each with different memory demands. Traditionally, communication between accelerators has been inefficient, typically orchestrated through explicit DMA transfers between different address spaces. More recently, industry has proposed unified coherent memory which enables implicit data movement and more data reuse, but often these interfaces limit the coherence flexibility available to heterogeneous systems. This paper demonstrates the benefits of fine-grained coherence specialization for heterogeneous systems. We propose an architecture that enables low-complexity independent specialization of each individual coherence request in heterogeneous workloads by building upon a simple and flexible baseline coherence interface, Spandex. We then describe how to optimize individual memory requests to improve cache reuse and performance-critical memory latency in emerging heterogeneous workloads. Collectively, our techniques enable significant gains, reducing execution time by up to 61% or network traffic by up to 99% while adding minimal complexity to the Spandex protocol.