ICP: Exploiting Instruction Correlation for Prefetching Irregular Memory Accesses

TL;DR

This paper introduces Instruction-Correlation Prefetching (ICP), a hardware mechanism that improves irregular memory access prefetching by leveraging instruction-level correlations rather than address recurrence, achieving significant performance gains.

Contribution

ICP is a novel prefetching approach that exploits instruction correlations to handle irregular memory accesses, reducing storage overhead and outperforming existing methods.

Findings

ICP outperforms state-of-the-art prefetchers by 14% on SPEC CPU benchmarks.

ICP requires only 2.1 KB of hardware storage, much less than traditional prefetchers.

ICP effectively prefetches irregular memory accesses by learning instruction dependencies.

Abstract

Irregular memory accesses pose challenges for effective and efficient data prefetching. While temporal prefetchers have recently shown promise for irregular memory access patterns, their effectiveness fundamentally depends on temporal address recurrence and large metadata storage. When memory addresses exhibit weak or no recurrence, as in indirect memory accesses, temporal prefetchers achieve limited performance gains while incurring substantial storage overhead. This paper proposes Instruction-Correlation Prefetching (ICP), a new hardware prefetching mechanism that exploits instruction-level correlations rather than memory-address correlations to handle irregular memory accesses. ICP observes that although memory addresses may not repeat, the instructions generating them often recur with stable data-dependency relationships. By learning these persistent instruction correlations, ICP speculatively computes and prefetches future irregular accesses using the execution results of their correlated predecessors. Across irregular SPEC CPU and GAP benchmarks, ICP outperforms the state-of-the-art temporal prefetcher Triangel by 14.0% and the indirect prefetcher DMP by 6.0%, while requiring only 2.1 KB of hardware storage, over three orders of magnitude smaller than temporal prefetchers.