ACIC: Admission-Controlled Instruction Cache

TL;DR

ACIC is a novel instruction cache design that uses admission control and prediction to better handle bursty instruction accesses in datacenter workloads, significantly improving performance over traditional caches.

Contribution

This paper introduces ACIC, a new cache architecture combining an i-Filter and temporal locality prediction to reduce pollution and enhance datacenter workload performance.

Findings

ACIC achieves 1.0223x average speedup over baseline LRU caches.

ACIC bridges over half the performance gap between LRU and optimal caching.

ACIC outperforms existing pollution reduction techniques in datacenter workloads.

Abstract

The front end bottleneck in datacenter workloads has come under increased scrutiny, with the growing code footprint, involvement of numerous libraries and OS services, and the unpredictability in the instruction stream. Our examination of these workloads points to burstiness in accesses to instruction blocks, which has also been observed in data accesses. Such burstiness is largely due to spatial and short-duration temporal localities, that LRU fails to recognize and optimize for, when a single cache caters to both forms of locality. Instead, we incorporate a small i-Filter as in previous works to separate spatial from temporal accesses. However, a simple separation does not suffice, and we additionally need to predict whether the block will continue to have temporal locality, after the burst of spatial locality. This combination of i-Filter and temporal locality predictor constitutes our Admission-Controlled Instruction Cache (ACIC). ACIC outperforms a number of state-of-the-art pollution reduction techniques (replacement algorithms, bypassing mechanisms, victim caches), providing 1.0223 speedup on the average over a baseline LRU based conventional i-cache (bridging over half of the gap between LRU and OPT) across several datacenter workloads.