Performance Health Index for Complex Cyber Infrastructures

TL;DR

This paper introduces a Configuration Health Index (CHI) framework to quantify how configuration variables impact system performance, providing better insights than existing models and avoiding pitfalls of purely data-driven approaches.

Contribution

The paper presents a novel CHI framework that effectively assesses configuration impact on performance, integrating domain knowledge with limited data for improved accuracy.

Findings

CHI outperforms segmented non-linear models

CHI provides more reliable performance insights

CHI avoids errors common in pure data-driven models

Abstract

Most IT systems depend on a set of configuration variables (CVs), expressed as a name/value pair that collectively define the resource allocation for the system. While the ill-effects of misconfiguration or improper resource allocation are well-known, there is no effective a priori metric to quantify the impact of the configuration on the desired system attributes such as performance, availability, etc. In this paper, we propose a \textit{Configuration Health Index} (CHI) framework specifically attuned to the performance attribute to capture the influence of CVs on the performance aspects of the system. We show how CHI, which is defined as a configuration scoring system, can take advantage of the domain knowledge and the available (but rather limited) performance data to produce important insights into the configuration settings. We compare the CHI with both well-advertised segmented non-linear models and state-of-the-art data-driven models, and show that the CHI not only consistently provides better results but also avoids the dangers of pure data drive approach which may predict incorrect behavior or eliminate some essential configuration variables from consideration.