Improved Prefetching Techniques for Linked Data Structures

TL;DR

This paper introduces Linkey, a prefetcher that uses programmer and compiler hints to improve prefetching accuracy and performance for linked data structures, significantly reducing cache misses and increasing IPC.

Contribution

The paper presents Linkey, a novel prefetcher that leverages layout hints to effectively prefetch linked data structures, outperforming traditional prefetching methods.

Findings

13% reduction in miss rate on average

65.4% increase in prefetching accuracy

1.40% average IPC improvement

Abstract

With ever-increasing main memory stall times, we need novel techniques to reduce effective memory access latencies. Prefetching has been shown to be an effective solution, especially with contiguous data structures that follow the traditional principles of spatial and temporal locality. However, on linked data structures$-$made up of many nodes linked together with pointers$-$typical prefetchers struggle, failing to predict accesses as elements are arbitrarily scattered throughout memory and access patters are arbitrarily complex and hence difficult to predict. To remedy these issues, we introduce $\textit{Linkey}$, a novel prefetcher that utilizes hints from the programmer/compiler to cache layout information and accurately prefetch linked data structures. $\textit{Linkey}$ obtains substantial performance improvements over a striding baseline. We achieve a geomean 13% reduction in miss rate with a maximum improvement of 58.8%, and a 65.4% geomean increase in accuracy, with many benchmarks improving from 0%. On benchmarks where $\textit{Linkey}$ is applicable, we observe a geomean IPC improvement of 1.40%, up to 12.1%.