Generating Unit Tests for Documentation

TL;DR

This paper introduces DScribe, a tool that leverages redundancy between tests and documentation to automatically generate and update consistent, checkable documentation and unit tests from a single source, improving quality and reducing inconsistencies.

Contribution

The paper presents DScribe, a novel template-based approach for automatic generation of unit tests and documentation from redundant artifacts, enhancing consistency and maintainability.

Findings

DScribe generated 97% of missing tests and documentation.

85% of specifications about exception handling were initially untested or undocumented.

Evaluation on Apache Commons IO showed high effectiveness of the approach.

Abstract

Software projects capture information in various kinds of artifacts, including source code, tests, and documentation. Such artifacts routinely encode information that is redundant, i.e., when a specification encoded in the source code is also separately tested and documented. Without supporting technology, such redundancy easily leads to inconsistencies and a degradation of documentation quality. We designed a tool-supported technique, called DScribe, that leverages redundancy between tests and documentation to generate consistent and checkable documentation and unit tests based on a single source of information. DScribe generates unit tests and documentation fragments based on a novel template and artifact generation technology. By pairing tests and documentation generation, DScribe provides a mechanism to automatically detect and replace outdated documentation. Our evaluation of the Apache Commons IO library revealed that of 835 specifications about exception handling, 85% of them were not tested or correctly documented, and DScribe could be used to automatically generate 97% of the tests and documentation.