Performance Measurements in the AI-Centric Computing Continuum Systems

TL;DR

This paper reviews existing performance metrics and discusses emerging dimensions like sustainability and energy efficiency in the evolving Distributed Computing Continuum, emphasizing the need for updated measurement standards in AI-centric systems.

Contribution

It provides a comprehensive review of current performance metrics and proposes considerations for selecting appropriate metrics in AI-driven distributed systems.

Findings

Traditional metrics need updating for AI workloads

Emerging dimensions include sustainability and energy efficiency

Guidelines for metric selection are outlined

Abstract

Over the Eight decades, computing paradigms have shifted from large, centralized systems to compact, distributed architectures, leading to the rise of the Distributed Computing Continuum (DCC). In this model, multiple layers such as cloud, edge, Internet of Things (IoT), and mobile platforms work together to support a wide range of applications. Recently, the emergence of Generative AI and large language models has further intensified the demand for computational resources across this continuum. Although traditional performance metrics have provided a solid foundation, they need to be revisited and expanded to keep pace with changing computational demands and application requirements. Accurate performance measurements benefit both system designers and users by supporting improvements in efficiency and promoting alignment with system goals. In this context, we review commonly used metrics in DCC and IoT environments. We also discuss emerging performance dimensions that address evolving computing needs, such as sustainability, energy efficiency, and system observability. We also outline criteria and considerations for selecting appropriate metrics, aiming to inspire future research and development in this critical area.