Fetch-Directed Instruction Prefetching Revisited

TL;DR

This paper enhances fetch-directed instruction prefetching by optimizing branch target buffer (BTB) storage, enabling larger reach and improved performance through multiple smaller BTBs, tag compression, and elimination of basic-block-specific storage.

Contribution

Introduces FDIP-X, a novel BTB architecture with multiple smaller BTBs, tag compression, and no basic-block size storage, significantly extending BTB reach and outperforming prior FDIP designs.

Findings

FDIP-X outperforms conventional FDIP with equal storage.

Multiple smaller BTBs reduce storage overhead.

Tag compression to 16 bits improves efficiency.

Abstract

Prior work has observed that fetch-directed prefetching (FDIP) is highly effective at covering instruction cache misses. The key to FDIP's effectiveness is having a sufficiently large BTB to accommodate the application's branch working set. In this work, we introduce several optimizations that significantly extend the reach of the BTB within the available storage budget. Our optimizations target nearly every source of storage overhead in each BTB entry; namely, the tag, target address, and size fields. We observe that while most dynamic branch instances have short offsets, a large number of branches has longer offsets or requires the use of full target addresses. Based on this insight, we break-up the BTB into multiple smaller BTBs, each storing offsets of different length. This enables a dramatic reduction in storage for target addresses. We further compress tags to 16 bits and avoid the use of the basic-block-oriented BTB advocated in prior FDIP variants. The latter optimization eliminates the need to store the basic block size in each BTB entry. Our final design, called FDIP-X, uses an ensemble of 4 BTBs and always outperforms conventional FDIP with a unified basic-block-oriented BTB for equal storage budgets.