SCOPE: C3SR Systems Characterization and Benchmarking Framework

TL;DR

The paper introduces SCOPE, a flexible benchmarking framework designed to simplify performance measurement development across high-performance computing systems, with tools for independent benchmark creation and visualization.

Contribution

It presents a novel software architecture that enables extensible, portable benchmarking and easy visualization, lowering barriers for developers at various expertise levels.

Findings

Provides a modular architecture for benchmarking

Includes C/C++ abstractions and utilities

Offers a Python package for visualization

Abstract

This report presents the design of the Scope infrastructure for extensible and portable benchmarking. Improvements in high- performance computing systems rely on coordination across different levels of system abstraction. Developing and defining accurate performance measurements is necessary at all levels of the system hierarchy, and should be as accessible as possible to developers with different backgrounds. The Scope project aims to lower the barrier to entry for developing performance benchmarks by providing a software architecture that allows benchmarks to be developed independently, by providing useful C/C++ abstractions and utilities, and by providing a Python package for generating publication-quality plots of resulting measurements.