Breakdown of a Benchmark Score Without Internal Analysis of Benchmarking Program

TL;DR

This paper introduces a novel method to analyze benchmark scores by using regression analysis on system data, eliminating the need for internal code analysis of benchmarking programs.

Contribution

It proposes a new approach to decompose benchmark scores without internal program analysis, reducing labor and dependency on the benchmarking code.

Findings

Successfully applied to 3 benchmarks on 15 Android smartphones.

Achieved effective score breakdown despite some accuracy limitations.

Reduces need for internal code analysis in benchmarking.

Abstract

A breakdown of a benchmark score is how much each aspect of the system performance affects the score. Existing methods require internal analysis on the benchmarking program and then involve the following problems: (1) require a certain amount of labor for code analysis, profiling, simulation, and so on and (2) require the benchmarking program itself. In this paper, we present a method for breaking down a benchmark score without internal analysis of the benchmarking program. The method utilizes regression analysis of benchmark scores on a number of systems. Experimental results with 3 benchmarks on 15 Android smartphones showed that our method could break down those benchmark scores even though there is room for improvement in accuracy.