Tapir: Automation Support of Exploratory Testing Using Model Reconstruction of the System Under Test

TL;DR

This paper introduces Tapir, a framework that automates and supports exploratory testing by reconstructing system models during testing, improving efficiency and defect detection compared to manual testing.

Contribution

The paper presents a novel framework that incrementally reconstructs system models during exploratory testing, automates navigation, and enhances testing efficiency and defect detection.

Findings

Increased system exploration extent with framework support

Improved time efficiency of testing process

Team-based navigational strategies further enhance efficiency

Abstract

For a considerable number of software projects, the creation of effective test cases is hindered by design documentation that is either lacking, incomplete or obsolete. The exploratory testing approach can serve as a sound method in such situations. However, the efficiency of this testing approach strongly depends on the method, the documentation of explored parts of a system, the organization and distribution of work among individual testers on a team, and the minimization of potential (very probable) duplicities in performed tests. In this paper, we present a framework for replacing and automating a portion of these tasks. A screen-flow-based model of the tested system is incrementally reconstructed during the exploratory testing process by tracking testers' activities. With additional metadata, the model serves for an automated navigation process for a tester. Compared with the exploratory testing approach, which is manually performed in two case studies, the proposed framework allows the testers to explore a greater extent of the tested system and enables greater detection of the defects present in the system. The results show that the time efficiency of the testing process improved with framework support. This efficiency can be increased by team-based navigational strategies that are implemented within the proposed framework, which is documented by another case study presented in this paper.