HINNPerf: Hierarchical Interaction Neural Network for Performance Prediction of Configurable Systems

TL;DR

HINNPerf is a hierarchical neural network model that accurately predicts system performance across configurations and offers insights into option interactions, aiding optimization and understanding of complex configurable systems.

Contribution

The paper introduces HINNPerf, a novel hierarchical neural network with regularization for improved performance prediction and interpretability in configurable systems.

Findings

Achieves 22.67% higher prediction accuracy than state-of-the-art methods.

Effectively models complex interactions among configuration options.

Provides interpretability insights into option significance and interactions.

Abstract

Modern software systems are usually highly configurable, providing users with customized functionality through various configuration options. Understanding how system performance varies with different option combinations is important to determine optimal configurations that meet specific requirements. Due to the complex interactions among multiple options and the high cost of performance measurement under a huge configuration space, it is challenging to study how different configurations influence the system performance. To address these challenges, we propose HINNPerf, a novel hierarchical interaction neural network for performance prediction of configurable systems. HINNPerf employs the embedding method and hierarchic network blocks to model the complicated interplay between configuration options, which improves the prediction accuracy of the method. Besides, we devise a hierarchical regularization strategy to enhance the model robustness. Empirical results on 10 real-world configurable systems show that our method statistically significantly outperforms state-of-the-art approaches by achieving average 22.67% improvement in prediction accuracy. In addition, combined with the Integrated Gradients method, the designed hierarchical architecture provides some insights about the interaction complexity and the significance of configuration options, which might help users and developers better understand how the configurable system works and efficiently identify significant options affecting the performance.