CPU Simulation with Ranked Set Sampling and Repeated Subsampling

TL;DR

This paper introduces a novel application of ranked set sampling and repeated subsampling in CPU simulation to improve the accuracy and confidence of representative region selection, significantly reducing error margins.

Contribution

It adapts ranked set sampling and a repeated subsampling scheme to computer architecture simulation, enhancing representativeness and reducing simulation error compared to traditional methods.

Findings

RSS reduces confidence interval width by up to 50%

Repeated subsampling lowers maximum error from 35% to 10%

Achieves average error below 2% in SPEC CPU 2017 simulations

Abstract

Computer system simulation studies routinely rely on executing a limited number of short application regions, since full end-to-end simulation is prohibitively time-consuming. To preserve representativeness, existing methods employ either random sampling or phase-based characterization to identify representative regions. In this work, we revisit random sampling in the context of computer architecture simulation. To assess how the confidence level varies with different micro-architectural configurations, we examine how the sample standard deviation relates to the sample mean. We show that the ranked set sampling (RSS) technique - well established in the statistical literature - maps naturally to architectural simulation and yields significantly tighter confidence intervals than simple random sampling. Across our experiments, RSS reduces the confidence interval width by up to 50%. We further introduce a repeated subsampling scheme that identifies representative simulation regions for future studies. For a fixed sample size, this approach reduces the maximum observed error from 35% to 10%. Evaluating two selection criteria, we find that more informed subsample selection provides additional accuracy gains. Overall, our method achieves an average error below 2% and a maximum error of 3.5% across individual SPEC CPU 2017 Integer applications when simulating 30 regions of 1 million instructions each.