Public Release and Validation of SPEC CPU2017 PinPoints

TL;DR

This paper presents the collection, validation, and public release of program phase checkpoints (pinballs) for SPEC CPU2017, enabling more efficient and accurate architectural simulation research.

Contribution

It provides a validated set of pinballs for SPEC CPU2017 and details the methodology, addressing issues with outdated data and errors in previous collections.

Findings

Pinballs have an average CPI error rate of 12%.

The methodology improves the accuracy and usability of phase-based sampling.

Public release supports community research and benchmarking efforts.

Abstract

Phase-based statistical sampling methods such as SimPoints have proven to be effective at dramatically reducing the long time for architectural simulators to run large workloads such as SPEC CPU2017. However, generating and validating them is a long and tenuous process. While checkpoints of program phases, or "pinballs", of SPEC CPU2017 have been collected by other researchers and shared with the research community, they are outdated and produce errors when used with the latest versions of the Sniper architectural simulator. To facilitate our own research as well as contribute to the community, we collect and validate our own pinballs for the SPEC CPU2017 SPECspeed suite and release them to the public domain. In this work we document our methodology, the hardware and software details of the collection process, and our validation results. In terms of CPI, our pinballs have an average error rate of 12% when compared with the native whole-program benchmark execution.