Measuring the Complexity of Ultra-Large-Scale Adaptive Systems

TL;DR

This paper introduces information-theoretic measures to evaluate and guide the design of ultra-large-scale adaptive systems, demonstrating their application through a computing system with dynamic workloads and adaptation strategies.

Contribution

It proposes novel complexity, emergence, self-organization, and homeostasis measures for ULS systems, enabling their evaluation and guiding adaptive design.

Findings

Highly unstable configurations correlate with high complexity variance.

Less aggressive adaptation yields more stability but may reduce optimal performance.

Measures effectively track system evolution under different workload conditions.

Abstract

Ultra-large scale (ULS) systems are becoming pervasive. They are inherently complex, which makes their design and control a challenge for traditional methods. Here we propose the design and analysis of ULS systems using measures of complexity, emergence, self-organization, and homeostasis based on information theory. These measures allow the evaluation of ULS systems and thus can be used to guide their design. We evaluate the proposal with a ULS computing system provided with adaptation mechanisms. We show the evolution of the system with stable and also changing workload, using different fitness functions. When the adaptive plan forces the system to converge to a predefined performance level, the nodes may result in highly unstable configurations, that correspond to a high variance in time of the measured complexity. Conversely, if the adaptive plan is less "aggressive", the system may be more stable, but the optimal performance may not be achieved.