Profile-Guided Temporal Prefetching

TL;DR

Prophet is a hardware-software co-designed framework that uses profile-guided methods to optimize metadata storage management, significantly improving the performance of temporal prefetching for irregular memory access patterns.

Contribution

It introduces a novel profile-guided approach that dynamically tunes metadata hints, enabling efficient coexistence with existing prefetchers and outperforming state-of-the-art solutions.

Findings

Prophet outperforms Triangel by 14.23% in performance.

Effective for complex temporal patterns where prior solutions fail.

Introduces negligible overhead in profiling and analysis.

Abstract

Temporal prefetching shows promise for handling irregular memory access patterns, which are common in data-dependent and pointer-based data structures. Recent studies introduced on-chip metadata storage to reduce the memory traffic caused by accessing metadata from off-chip DRAM. However, existing prefetching schemes struggle to efficiently utilize the limited on-chip storage. An alternative solution, software indirect access prefetching, remains ineffective for optimizing temporal prefetching. In this work, we propose Prophet--a hardware-software co-designed framework that leverages profile-guided methods to optimize metadata storage management. Prophet profiles programs using counters instead of traces, injects hints into programs to guide metadata storage management, and dynamically tunes these hints to enable the optimized binary to adapt to different program inputs. Prophet is designed to coexist with existing hardware temporal prefetchers, delivering efficient, high-performance solutions for frequently executed workloads while preserving the original runtime scheme for less frequently executed workloads. Prophet outperforms the state-of-the-art temporal prefetcher, Triangel, by 14.23%, effectively addressing complex temporal patterns where prior profile-guided solutions fall short (only achieving 0.1% performance gain). Prophet delivers superior performance across all evaluated workload inputs, introducing negligible profiling, analysis, and instruction overhead.