Benchmarking for Single Feature Attribution with Microarchitecture Cliffs

TL;DR

This paper introduces Microarchitecture Cliffs, a benchmark generation methodology that improves simulator calibration by isolating microarchitectural features, significantly reducing performance prediction errors in architectural simulators.

Contribution

The paper presents a novel benchmark generation methodology and automated tools for precise calibration of simulators against RTL, enhancing accuracy in microarchitectural performance modeling.

Findings

Reduced simulator performance error from 59.2% to 1.4%.

Achieved 48.03% reduction in feature-specific error.

Lowered overall performance errors on SPEC benchmarks.

Abstract

Architectural simulators play a critical role in early microarchitectural exploration due to their flexibility and high productivity. However, their effectiveness is often constrained by fidelity: simulators may deviate from the behavior of the final RTL, leading to unreliable performance estimates. Consequently, model calibration, which aligns simulator behavior with the RTL as the ground-truth microarchitecture, becomes essential for achieving accurate performance modeling. To facilitate model calibration accuracy, we propose Microarchitecture Cliffs, a benchmark generation methodology designed to expose mismatches in microarchitectural behavior between the simulator and RTL. After identifying the key architectural components that require calibration, the Cliff methodology enables precise attribution of microarchitectural differences to a single microarchitectural feature through a set of benchmarks. In addition, we develop a set of automated tools to improve the efficiency of the Cliff workflow. We apply the Cliff methodology to calibrate the XiangShan version of gem5 (XS-GEM5) against the XiangShan open-source CPU (XS-RTL). We reduce the performance error of XS-GEM5 from 59.2% to just 1.4% on the Cliff benchmarks. Meanwhile, the calibration guided by Cliffs effectively reduces the relative error of a representative tightly coupled microarchitectural feature by 48.03%. It also substantially lowers the absolute performance error, with reductions of 15.1% and 21.0% on SPECint2017 and SPECfp2017, respectively.