Testing Real-World Healthcare IoT Application: Experiences and Lessons Learned

TL;DR

This paper evaluates a REST API testing approach (RESTest) on a real-world healthcare IoT application, highlighting its effectiveness in discovering failures and faults, and sharing practical lessons learned from the industrial case study.

Contribution

It presents an industrial evaluation of RESTest on healthcare IoT, demonstrating its effectiveness and providing insights and lessons for testing complex healthcare systems.

Findings

RESTest achieved approximately 56% API coverage.

Several failures were identified across REST APIs.

Nine potential faults were detected.

Abstract

Healthcare Internet of Things (IoT) applications require rigorous testing to ensure their dependability. Such applications are typically integrated with various third-party healthcare applications and medical devices through REST APIs. This integrated network of healthcare IoT applications leads to REST APIs with complicated and interdependent structures, thus creating a major challenge for automated system-level testing. We report an industrial evaluation of a state-of-the-art REST APIs testing approach (RESTest) on a real-world healthcare IoT application. We analyze the effectiveness of RESTest's testing strategies regarding REST APIs failures, faults in the application, and REST API coverage, by experimenting with six REST APIs of 41 API endpoints of the healthcare IoT application. Results show that several failures are discovered in different REST APIs with ~56% coverage using RESTest. Moreover, nine potential faults are identified. Using the evidence collected from the experiments, we provide our experiences and lessons learned.