Model-based testing in practice: An experience report from the web applications domain

TL;DR

This paper reports on the practical application of model-based testing (MBT) in a large software testing company, demonstrating its benefits in test coverage, design, and fault detection through an industry case study.

Contribution

It provides empirical evidence and practical insights on deploying MBT in industrial settings, highlighting challenges, benefits, and lessons learned.

Findings

Improved test coverage and fault detection with MBT

Enhanced test design practices through MBT

Practical challenges and solutions in applying MBT

Abstract

In the context of a large software testing company, we have deployed the model-based testing (MBT) approach to take the company's test automation practices to higher levels of maturity /and capability. We have chosen, from a set of open-source/commercial MBT tools, an open-source tool named GraphWalker, and have pragmatically used MBT for end-to-end test automation of several large web and mobile applications under test. The MBT approach has provided, so far in our project, various tangible and intangible benefits in terms of improved test coverage (number of paths tested), improved test-design practices, and also improved real-fault detection effectiveness. The goal of this experience report (applied research report), done based on "action research", is to share our experience of applying and evaluating MBT as a software technology (technique and tool) in a real industrial setting. We aim at contributing to the body of empirical evidence in industrial application of MBT by sharing our industry-academia project on applying MBT in practice, the insights that we have gained, and the challenges and questions that we have faced and tackled so far. We discuss an overview of the industrial setting, provide motivation, explain the events leading to the outcomes, discuss the challenges faced, summarize the outcomes, and conclude with lessons learned, take-away messages, and practical advices based on the described experience. By learning from the best practices in this paper, other test engineers could conduct more mature MBT in their test projects.