Extracting Clean Performance Models from Tainted Programs

TL;DR

This paper introduces a novel approach using taint analysis to improve empirical performance modeling of parallel applications, reducing costs and enhancing model accuracy amidst noisy data and complex experiment design.

Contribution

It demonstrates how taint analysis can significantly enhance the process of deriving performance models from empirical data, addressing challenges like noise and experiment complexity.

Findings

Taint analysis reduces the cost of performance modeling.

It improves the accuracy of empirical performance models.

The approach helps validate experimental setups.

Abstract

Performance models are well-known instruments to understand the scaling behavior of parallel applications. They express how performance changes as key execution parameters, such as the number of processes or the size of the input problem, vary. Besides reasoning about program behavior, such models can also be automatically derived from performance data. This is called empirical performance modeling. While this sounds simple at the first glance, this approach faces several serious interrelated challenges, including expensive performance measurements, inaccuracies inflicted by noisy benchmark data, and overall complex experiment design, starting with the selection of the right parameters. The more parameters one considers, the more experiments are needed and the stronger the impact of noise. In this paper, we show how taint analysis, a technique borrowed from the domain of computer security, can substantially improve the modeling process, lowering its cost, improving model quality, and help validate performance models and experimental setups.