White-Box Analysis over Machine Learning: Modeling Performance of Configurable Systems

TL;DR

This paper introduces Comprex, a white-box method for modeling system performance influenced by configurations, achieving accuracy comparable to black-box methods but with lower measurement costs and enhanced interpretability.

Contribution

Comprex combines local measurements, dynamic taint analysis, and configuration space compression to build accurate, interpretable performance models without machine learning extrapolation.

Findings

Comprex achieves similar accuracy to expensive black-box models.

Comprex reduces measurement effort and cost.

Models are interpretable and locally insightful.

Abstract

Performance-influence models can help stakeholders understand how and where configuration options and their interactions influence the performance of a system. With this understanding, stakeholders can debug performance behavior and make deliberate configuration decisions. Current black-box techniques to build such models combine various sampling and learning strategies, resulting in tradeoffs between measurement effort, accuracy, and interpretability. We present Comprex, a white-box approach to build performance-influence models for configurable systems, combining insights of local measurements, dynamic taint analysis to track options in the implementation, compositionality, and compression of the configuration space, without relying on machine learning to extrapolate incomplete samples. Our evaluation on 4 widely-used, open-source projects demonstrates that Comprex builds similarly accurate performance-influence models to the most accurate and expensive black-box approach, but at a reduced cost and with additional benefits from interpretable and local models.