Coverage measurement in model-based testing of web applications: Tool support and an industrial experience report

TL;DR

This paper introduces MBTCover, an open-source tool for live measurement of multiple coverage types in model-based testing of web applications, addressing a gap in existing tools and sharing industrial experience.

Contribution

The paper presents MBTCover, a novel open-source tool that integrates various coverage metrics in real-time during web application testing with MBT.

Findings

Successfully used in large industrial projects

Enabled live monitoring of multiple coverage types

Facilitated integration of front-end, back-end, and requirements coverage

Abstract

There are many widely used tools for measuring test-coverage and code-coverage. Test coverage is the ratio of requirements or other non-code artifacts covered by a test suite, while code-coverage is the ratio of source code covered by tests. Almost all coverage tools show a few certain subset of coverage values, and almost always either test-coverage or code-coverage measures. In a large-scale industrial web-application-testing setting, we were faced with the need to "integrate" several types of coverage data (including front-end and back-end code coverage with requirements coverage), and to see all of them "live" as large model-based test suites were running. By being unable to find any off-the-shelf toolset to address the above need, we have developed an open-source test coverage tool, specific for MBT, named MBTCover. In addition to code coverage, the tool measures and reports requirements and model coverage, "live" as a given MBT test suite is executing. In this paper, we present the features of the MBTCover tool and our experience from using it in multiple large test-automation projects in practice. Other software test engineers, who conduct web application testing and MBT, may find the tool useful in their projects.