Continuously Testing Distributed IoT Systems: An Overview of the State of the Art

TL;DR

This paper reviews the current state of continuous testing for IoT systems, highlighting challenges, evaluating 18 tools, and discussing future research directions in IoT testing environments.

Contribution

It provides a comprehensive overview of IoT testing challenges, compares existing tools, and outlines future trends in IoT test environment development.

Findings

Identifies key characteristics of effective IoT test environments.

Provides a comparative analysis of 18 IoT testing tools.

Discusses future research directions in IoT testing.

Abstract

The continuous testing of small changes to systems has proven to be useful and is widely adopted in the development of software systems. For this, software is tested in environments that are as close as possible to the production environments. When testing IoT systems, this approach is met with unique challenges that stem from the typically large scale of the deployments, heterogeneity of nodes, challenging network characteristics, and tight integration with the environment among others. IoT test environments present a possible solution to these challenges by emulating the nodes, networks, and possibly domain environments in which IoT applications can be executed. This paper gives an overview of the state of the art in IoT testing. We derive desirable characteristics of IoT test environments, compare 18 tools that can be used in this respect, and give a research outlook of future trends in this area.