Differential coverage: automating coverage analysis

TL;DR

This paper introduces differential coverage and date binning methods to automate coverage data analysis, reducing manual effort and making it accessible for large-scale projects across various languages and types.

Contribution

It presents novel methods for combining coverage data with project history, implemented in the open-source tool gendiffcov, to streamline coverage analysis.

Findings

Applicable to any coverage metric and language

Reduces manual effort in coverage analysis

Implemented in the open-source tool gendiffcov

Abstract

While it is easy to automate coverage data collection, it is a time consuming/difficult/expensive manual process to analyze the data so that it can be acted upon. Complexity arises from numerous sources, of which untested or poorly tested legacy code and third-party libraries are two of the most common. Differential coverage and date binning are methods of combining coverage data and project/file history to determine if goals have been met and to identify areas of code which should be prioritized. These methods can be applied to any coverage metric which can be associated with a location -- statement, function, expression, toggle, etc. -- and to any language, including both software (C++, Python, etc.) and hardware description languages (SystemVerilog, VHDL). The goal of these approaches is to reduce the cost and the barrier to entry of using coverage data analysis in large-scale projects. The approach is realized in gendiffcov, a recently released open-source tool.