Fine-Grained Address Segmentation for Attention-Based Variable-Degree Prefetching

TL;DR

This paper introduces TransFetch, an attention-based model utilizing fine-grained address segmentation and delta bitmaps to improve memory prefetching accuracy and performance, significantly outperforming existing methods.

Contribution

The paper proposes a novel prefetching model, TransFetch, that effectively models address prediction using address segmentation and attention mechanisms, addressing large vocabulary issues.

Findings

Address segmentation improves F1-score by up to 36%.

TransFetch achieves 38.75% IPC improvement over no prefetching.

Outperforms rule-based BOP and ML-based Voyager prefetchers by over 6%.

Abstract

Machine learning algorithms have shown potential to improve prefetching performance by accurately predicting future memory accesses. Existing approaches are based on the modeling of text prediction, considering prefetching as a classification problem for sequence prediction. However, the vast and sparse memory address space leads to large vocabulary, which makes this modeling impractical. The number and order of outputs for multiple cache line prefetching are also fundamentally different from text prediction. We propose TransFetch, a novel way to model prefetching. To reduce vocabulary size, we use fine-grained address segmentation as input. To predict unordered sets of future addresses, we use delta bitmaps for multiple outputs. We apply an attention-based network to learn the mapping between input and output. Prediction experiments demonstrate that address segmentation achieves 26% - 36% higher F1-score than delta inputs and 15% - 24% higher F1-score than page & offset inputs for SPEC 2006, SPEC 2017, and GAP benchmarks. Simulation results show that TransFetch achieves 38.75% IPC improvement compared with no prefetching, outperforming the best-performing rule-based prefetcher BOP by 10.44%, and ML-based prefetcher Voyager by 6.64%.