Data Motif-based Proxy Benchmarks for Big Data and AI Workloads

TL;DR

This paper introduces a machine learning-based methodology to create data motif-driven proxy benchmarks that accurately mimic big data and AI workloads, significantly reducing simulation time while maintaining high fidelity.

Contribution

It presents a novel approach to construct practical proxy benchmarks from data motifs using machine learning, enabling efficient and accurate simulation of complex workloads.

Findings

Proxy benchmarks reduce execution time by hundreds of times.

Maintain over 90% accuracy in performance data.

Reflect consistent performance trends across architectures.

Abstract

For the architecture community, reasonable simulation time is a strong requirement in addition to performance data accuracy. However, emerging big data and AI workloads are too huge at binary size level and prohibitively expensive to run on cycle-accurate simulators. The concept of data motif, which is identified as a class of units of computation performed on initial or intermediate data, is the first step towards building proxy benchmark to mimic the real-world big data and AI workloads. However, there is no practical way to construct a proxy benchmark based on the data motifs to help simulation-based research. In this paper, we embark on a study to bridge the gap between data motif and a practical proxy benchmark. We propose a data motif-based proxy benchmark generating methodology by means of machine learning method, which combine data motifs with different weights to mimic the big data and AI workloads. Furthermore, we implement various data motifs using light-weight stacks and apply the methodology to five real-world workloads to construct a suite of proxy benchmarks, considering the data types, patterns, and distributions. The evaluation results show that our proxy benchmarks shorten the execution time by 100s times on real systems while maintaining the average system and micro-architecture performance data accuracy above 90%, even changing the input data sets or cluster configurations. Moreover, the generated proxy benchmarks reflect consistent performance trends across different architectures. To facilitate the community, we will release the proxy benchmarks on the project homepage http://prof.ict.ac.cn/BigDataBench.