Annotating Control-Flow Graphs for Formalized Test Coverage Criteria

TL;DR

This paper introduces a method to annotate control flow graphs with decision information, enabling formal definitions of coverage criteria like MC/DC, and provides a tool for automatic annotation from compiler outputs.

Contribution

It proposes the Control-Flow Decision Graph (CFDG) model to formalize coverage criteria and implements an algorithm for automatic CFG annotation from compiler outputs.

Findings

CFDG effectively captures decision points in CFGs.

The tool automates CFG annotation for coverage analysis.

Formal definitions improve understanding of coverage criteria.

Abstract

Control flow coverage criteria are an important part of the process of qualifying embedded software for safety-critical systems. Criteria such as modified condition/decision coverage (MC/DC) as defined by DO-178B are used by regulators to judge the adequacy of testing and by QA engineers to design tests when full path coverage is impossible. Despite their importance, these coverage criteria are often misunderstood. One problem is that their definitions are typically written in natural language specification documents, making them imprecise. Other works have proposed formal definitions using binary predicate logic, but these definitions are difficult to apply to the analysis of real programs. Control-Flow Graphs (CFGs) are the most common model for analyzing program logic in compilers, and seem to be a good fit for defining and analyzing coverage criteria. However, CFGs discard the explicit concept of a decision, making their use for this task seem impossible. In this paper, we show how to annotate a CFG with decision information inferred from the graph itself. We call this annotated model a Control-Flow Decision Graph (CFDG) and we use it to formally define several common coverage criteria. We have implemented our algorithms in a tool which we show can be applied to automatically annotate CFGs output from popular compilers.