PAI: Fast, Accurate, and Full Benchmark Performance Projection with AI

TL;DR

PAI is a novel AI-based method that rapidly and accurately predicts full benchmark performance using a hierarchical LSTM model, significantly reducing simulation time while maintaining high accuracy.

Contribution

It introduces the first full benchmark performance prediction technique that avoids detailed simulation and instruction-level encoding, leveraging a hierarchical LSTM model for efficiency and accuracy.

Findings

Achieves 9.35% IPC prediction error on SPEC CPU 2017.

Predicts full benchmark performance in under 3 minutes.

Requires three orders of magnitude less time than prior methods.

Abstract

The exponential increase in complex IPs within modern SoCs, driven by Moore's Law, has created a pressing need for fast and accurate hardware-software power-performance analysis. Traditional performance simulators (such as cycle accurate simulators) are often too slow to simulate full benchmarks within a reasonable timeframe; require considerable effort for development, maintenance, and extensions; and are prone to errors, making pre-silicon performance projections and competitive analysis increasingly challenging. Prior attempts in addressing this challenge using machine learning fall short as they are either slow, inaccurate or unable to predict the performance of full benchmarks. To address these limitations, we present PAI, the first technique to accurately predict full benchmark performance without relying on detailed simulation or instruction-wise encoding. At the heart of PAI is a hierarchical Long Short Term Memory (LSTM)-based model that takes a trace of microarchitecture independent features from a program execution and predicts performance metrics. We present the detailed design, implementation and evaluation of PAI. Our initial experiments showed that PAI can achieve an average IPC prediction error of 9.35% for SPEC CPU 2017 benchmark suite while taking only 2 min 57 sec for the entire suite. This prediction error is comparable to prior state-of-the-art techniques while requiring 3 orders of magnitude less time.