Quality and Reliability Metrics for IoT Systems: A Consolidated View

TL;DR

This paper consolidates and categorizes quality and reliability metrics specific to IoT systems, providing guidance on their application during development and testing phases.

Contribution

It offers the first comprehensive overview of IoT-specific quality and reliability metrics, including their categorization and recommended usage in testing processes.

Findings

Metrics are categorized into three groups: system quality, testing effectiveness, and universal metrics.

Guidelines for proper metric application in IoT testing are provided.

The study fills a gap by focusing specifically on IoT system metrics.

Abstract

Quality and reliability metrics play an important role in the evaluation of the state of a system during the development and testing phases, and serve as tools to optimize the testing process or to define the exit or acceptance criteria of the system. This study provides a consolidated view on the available quality and reliability metrics applicable to Internet of Things (IoT) systems, as no comprehensive study has provided such a view specific to these systems. The quality and reliability metrics categorized and discussed in this paper are divided into three categories: metrics assessing the quality of an IoT system or service, metrics for assessing the effectiveness of the testing process, and metrics that can be universally applied in both cases. In the discussion, recommendations of proper usage of discussed metrics in a testing process are then given.