Gaze into the Pattern: Characterizing Spatial Patterns with Internal Temporal Correlations for Hardware Prefetching

TL;DR

Gaze is a hardware prefetcher that leverages internal temporal correlations within spatial memory access patterns to improve prefetching efficiency and performance across various scenarios.

Contribution

The paper introduces Gaze, a novel prefetching method that utilizes footprint-internal temporal correlations, addressing limitations of context-based predictions.

Findings

Gaze improves single-core performance by up to 5.7%.

Gaze enhances eight-core performance by up to 11.4%.

Gaze outperforms recent solutions PMP and vBerti.

Abstract

Hardware prefetching is one of the most widely-used techniques for hiding long data access latency. To address the challenges faced by hardware prefetching, architects have proposed to detect and exploit the spatial locality at the granularity of spatial region. When a new region is activated, they try to find similar previously accessed regions for footprint prediction based on system-level environmental features such as the trigger instruction or data address. However, we find that such context-based prediction cannot capture the essential characteristics of access patterns, leading to limited flexibility, practicality and suboptimal prefetching performance. In this paper, inspired by the temporal property of memory accessing, we note that the temporal correlation exhibited within the spatial footprint is a key feature of spatial patterns. To this end, we propose Gaze, a simple and efficient hardware spatial prefetcher that skillfully utilizes footprint-internal temporal correlations to efficiently characterize spatial patterns. Meanwhile, we observe a unique unresolved challenge in utilizing spatial footprints generated by spatial streaming, which exhibit extremely high access density. Therefore, we further enhance Gaze with a dedicated two-stage approach that mitigates the over-prefetching problem commonly encountered in conventional schemes. Our comprehensive and diverse set of experiments show that Gaze can effectively enhance the performance across a wider range of scenarios. Specifically, Gaze improves performance by 5.7\% and 5.4\% at single-core, 11.4\% and 8.8\% at eight-core, compared to most recent low-cost solutions PMP and vBerti.