Reuse Cache for Heterogeneous CPU-GPU Systems

TL;DR

This paper introduces a reuse cache for heterogeneous CPU-GPU systems that improves cache efficiency by storing only frequently reused data, achieving near-ideal IPC gains with significantly reduced area costs.

Contribution

The paper proposes a novel reuse cache design tailored for heterogeneous CPU-GPU systems, addressing their unique cache access patterns and improving efficiency.

Findings

Reuse cache achieves within 0.5% of IPC gains of a static partitioned LLC.

Reduces LLC area cost by an average of 40%.

Effectively handles different data access patterns in CPU-GPU systems.

Abstract

It is generally observed that the fraction of live lines in shared last-level caches (SLLC) is very small for chip multiprocessors (CMPs). This can be tackled using promotion-based replacement policies like re-reference interval prediction (RRIP) instead of LRU, dead-block predictors, or reuse-based cache allocation schemes. In GPU systems, similar LLC issues are alleviated using various cache bypassing techniques. These issues are worsened in heterogeneous CPU-GPU systems because the two processors have different data access patterns and frequencies. GPUs generally work on streaming data, but have many more threads accessing memory as compared to CPUs. As such, most traditional cache replacement and allocation policies prove ineffective due to the higher number of cache accesses in GPU applications, resulting in higher allocation for GPU cache lines, despite their minimal reuse. In this work, we implement the Reuse Cache approach for heterogeneous CPU-GPU systems. The reuse cache is a decoupled tag/data SLLC which is designed to only store the data that is being accessed more than once. This design is based on the observation that most of the cache lines in the LLC are stored but do not get reused before being replaced. We find that the reuse cache achieves within 0.5% of the IPC gains of a statically partitioned LLC, while decreasing the area cost of the LLC by an average of 40%.