Tracing Execution of Software for Design Coverage

TL;DR

This paper introduces a generic approach to trace software execution for design coverage, helping test engineers understand noncovered parts and improve test suite development using design abstractions and trace analysis.

Contribution

It proposes a novel method for design coverage of executed software, integrating with Third Eye tracing to enhance test suite creation and understanding.

Findings

Enables understanding of noncovered design elements.

Supports various design abstractions like statecharts and activity diagrams.

Facilitates faster test suite development.

Abstract

Test suites are designed to validate the operation of a system against requirements. One important aspect of a test suite design is to ensure that system operation logic is tested completely. A test suite should drive a system through all abstract states to exercise all possible cases of its operation. This is a difficult task. Code coverage tools support test suite designers by providing the information about which parts of source code are covered during system execution. Unfortunately, code coverage tools produce only source code coverage information. For a test engineer it is often hard to understand what the noncovered parts of the source code do and how they relate to requirements. We propose a generic approach that provides design coverage of the executed software simplifying the development of new test suites. We demonstrate our approach on common design abstractions such as statecharts, activity diagrams, message sequence charts and structure diagrams. We implement the design coverage using Third Eye tracing and trace analysis framework. Using design coverage, test suites could be created faster by focussing on untested design elements.