Memory Centric Characterization and Analysis of SPEC CPU2017 Suite

TL;DR

This paper provides a detailed memory-centric analysis of the SPEC CPU2017 benchmark suite, revealing its high memory intensity, variable memory profiles, and increased instruction and compute demands compared to previous benchmarks.

Contribution

It offers the first comprehensive memory-centric characterization of SPEC CPU2017, including insights into memory usage, bandwidth, and instruction distribution.

Findings

SPEC CPU2017 workloads are approximately 50% memory intensive.

Memory footprint varies widely, with some benchmarks using up to 16 GB of memory.

Instruction count is an order of magnitude higher than in SPEC CPU2006.

Abstract

In this paper we provide a comprehensive, memory-centric characterization of the SPEC CPU2017 benchmark suite, using a number of mechanisms including dynamic binary instrumentation, measurements on native hardware using hardware performance counters and OS based tools. We present a number of results including working set sizes, memory capacity consumption and, memory bandwidth utilization of various workloads. Our experiments reveal that the SPEC CPU2017 workloads are surprisingly memory intensive, with approximately 50% of all dynamic instructions being memory intensive ones. We also show that there is a large variation in the memory footprint and bandwidth utilization profiles of the entire suite, with some benchmarks using as much as 16 GB of main memory and up to 2.3 GB/s of memory bandwidth. We also perform instruction execution and distribution analysis of the suite and find that the average instruction count for SPEC CPU2017 workloads is an order of magnitude higher than SPEC CPU2006 ones. In addition, we also find that FP benchmarks of the SPEC 2017 suite have higher compute requirements: on average, FP workloads execute three times the number of compute operations as compared to INT workloads.