An Efficient Method for Uncertainty Propagation in Robust Software Performance Estimation

TL;DR

This paper introduces Polynomial Chaos Expansion (PCE) as an efficient alternative to Monte Carlo methods for uncertainty propagation in software performance estimation, significantly reducing computational costs while maintaining high accuracy.

Contribution

The paper extends PCE to robust performance estimation in software systems, demonstrating its efficiency and accuracy across diverse case studies.

Findings

PCE accurately estimates system robustness with over 97% accuracy.

PCE reduces performance evaluation time by up to 225 hours compared to Monte Carlo methods.

The approach is effective across different development phases and application domains.

Abstract

Software engineers often have to estimate the performance of a software system before having full knowledge of the system parameters, such as workload and operational profile. These uncertain parameters inevitably affect the accuracy of quality evaluations, and the ability to judge if the system can continue to fulfil performance requirements if parameter results are different from expected. Previous work has addressed this problem by modelling the potential values of uncertain parameters as probability distribution functions, and estimating the robustness of the system using Monte Carlo-based methods. These approaches require a large number of samples, which results in high computational cost and long waiting times. To address the computational inefficiency of existing approaches, we employ Polynomial Chaos Expansion (PCE) as a rigorous method for uncertainty propagation and further extend its use to robust performance estimation. The aim is to assess if the software system is robust, i.e., it can withstand possible changes in parameter values, and continue to meet performance requirements. PCE is a very efficient technique, and requires significantly less computations to accurately estimate the distribution of performance indices. Through three very different case studies from different phases of software development and heterogeneous application domains, we show that PCE can accurately (>97\%) estimate the robustness of various performance indices, and saves up to 225 hours of performance evaluation time when compared to Monte Carlo Simulation.