Faster Discovery of Faster System Configurations with Spectral Learning

TL;DR

This paper introduces the WHAT spectral learner, which uses eigenvalues of configuration distance matrices to efficiently predict system performance with fewer samples and higher stability, enabling faster discovery of optimal configurations.

Contribution

The paper presents the WHAT spectral learner, a novel method that reduces sample requirements and improves stability in performance prediction for configurable software systems.

Findings

Achieves less than 10% prediction error with fewer than 10 samples

Requires 2 to 10 times fewer samples than existing methods

Predictive models effectively guide optimization towards near-optimal configurations

Abstract

Despite the huge spread and economical importance of configurable software systems, there is unsatisfactory support in utilizing the full potential of these systems with respect to finding performance-optimal configurations. Prior work on predicting the performance of software configurations suffered from either (a) requiring far too many sample configurations or (b) large variances in their predictions. Both these problems can be avoided using the WHAT spectral learner. WHAT's innovation is the use of the spectrum (eigenvalues) of the distance matrix between the configurations of a configurable software system, to perform dimensionality reduction. Within that reduced configuration space, many closely associated configurations can be studied by executing only a few sample configurations. For the subject systems studied here, a few dozen samples yield accurate and stable predictors - less than 10% prediction error, with a standard deviation of less than 2%. When compared to the state of the art, WHAT (a) requires 2 to 10 times fewer samples to achieve similar prediction accuracies, and (b) its predictions are more stable (i.e., have lower standard deviation). Furthermore, we demonstrate that predictive models generated by WHAT can be used by optimizers to discover system configurations that closely approach the optimal performance.