Model-based Automated Testing of Mobile Applications: An Industrial Case Study

TL;DR

This paper presents a model-based automated testing approach for mobile apps that uses cloud-hosted real devices, achieving high coverage and practical industrial application insights.

Contribution

It introduces a novel model-based test generation method executed on real devices, demonstrating its effectiveness in industrial mobile app testing.

Findings

High coverage of user interaction code achieved

Test generation on real devices increases confidence in app quality

External service integration needed for full application coverage

Abstract

Automatic testing of mobile applications has been a well-researched area in recent years. However, testing in industry is still a very manual practice, as research results have not been fully transferred and adopted. Considering mobile applications, manual testing has the additional burden of adequate testing posed by a large number of available devices and different configurations, as well as the maintenance and setup of such devices. In this paper, we propose and evaluate the use of a model-based test generation approach, where generated tests are executed on a set of cloud-hosted real mobile devices. By using a model-based approach we generate dynamic, less brittle, and implementation simple test cases. The test execution on multiple real devices with different configurations increase the confidence in the implementation of the system under test. Our evaluation shows that the used approach produces a high coverage of the parts of the application related to user interactions. Nevertheless, the inclusion of external services in test generation is required in order to additionally increase the coverage of the complete application. Furthermore, we present the lessons learned while transferring and implementing this approach in an industrial context and applying it to the real product.