JustinANN: Realistic Test Generation for Java Programs Driven by Annotations

TL;DR

JustinANN is a scalable tool that leverages Java annotations to generate realistic test inputs and assertions, improving test quality and defect detection in Java programs.

Contribution

The paper introduces a systematic annotation set and combination rules to generate meaningful test cases aligned with program specifications.

Findings

Compatible with existing annotations

Easier to generate test data within and outside requirement boundaries

Helps in detecting program defects

Abstract

Automated test case generation is important. However, the automatically generated test input does not always make sense, and the automated assertion is difficult to validate against the program under test. In this paper, we propose JustinANN, a flexible and scalable tool to generate test cases for Java programs, providing realistic test inputs and assertions. We have observed that, in practice, Java programs contain a large number of annotations from programs, which can be considered as part of the user specification. We design a systematic annotation set with 7 kinds of annotations and 4 combination rules based on them to modify complex Java objects. Annotations that modify the fields or return variables of methods can be used to generate assertions that represent the true intent of the program, and the ones that modify the input parameters can be used to generate test inputs that match the real business requirement. We have conducted experiments to evaluate the approach on open source Java programs. The results show that the annotations and their combinations designed in this paper are compatible with existing annotations; our approach is easier to generate test data in, on and outside the boundaries of the requirement domain; and it also helps to find program defects.