Generic and Effective Specification of Structural Test Objectives

TL;DR

This paper introduces HTOL, a unified formal language for specifying diverse structural test objectives, enabling encoding, measuring, and supporting various criteria within a single framework, thus enhancing test automation.

Contribution

The paper presents HTOL, a formal, generic specification language with a novel semantics and hyperlabels, allowing comprehensive and flexible expression of test objectives across different criteria.

Findings

HTOL can encode numerous existing test coverage criteria.

The coverage measurement tool for HTOL is efficient and practical.

Initial experiments demonstrate the approach's effectiveness.

Abstract

While a wide range of different, sometimes heterogeneous test coverage criteria have been proposed, there exists no generic formalism to describe them, and available test automation tools usually support only a small subset of them. We introduce a unified specification language, called HTOL, providing a powerful generic mechanism to define test objectives, which permits encoding numerous existing criteria and supporting them in a unified way. HTOL comes with a formal semantics and can express complex requirements over several executions (using a novel notion of hyperlabels), as well as alternative requirements or requirements over a whole program execution. A novel classification of a large class of existing criteria is proposed. Finally, a coverage measurement tool for HTOL objectives has been implemented. Initial experiments suggest that the proposed approach is both efficient and practical.