Observation-Enhanced QoS Analysis of Component-Based Systems

TL;DR

This paper introduces an observation-based refinement method for CTMC models to improve the accuracy of QoS analysis in component-based systems, significantly reducing errors compared to traditional approaches.

Contribution

It presents a novel model refinement technique using system observations, along with an automated tool and case studies demonstrating improved QoS prediction accuracy.

Findings

Reduced QoS analysis errors by up to 97%

Traditional CTMC analysis can be highly inaccurate

Method validated on real-world service and support systems

Abstract

We present a new method for the accurate analysis of the quality-of-service (QoS) properties of component-based systems. Our method takes as input a QoS property of interest and a high-level continuous-time Markov chain (CTMC) model of the analysed system, and refines this CTMC based on observations of the execution times of the system components. The refined CTMC can then be analysed with existing probabilistic model checkers to accurately predict the value of the QoS property. The paper describes the theoretical foundation underlying this model refinement, the tool we developed to automate it, and two case studies that apply our QoS analysis method to a service-based system implemented using public web services and to an IT support system at a large university, respectively. Our experiments show that traditional CTMC-based QoS analysis can produce highly inaccurate results and may lead to invalid engineering and business decisions. In contrast, our new method reduced QoS analysis errors by 84.4-89.6% for the service-based system and by 94.7-97% for the IT support system, significantly lowering the risk of such invalid decisions.