A3Test: Assertion-Augmented Automated Test Case Generation

TL;DR

A3Test is a DL-based approach that significantly improves automated test case generation by incorporating assertion knowledge and verification mechanisms, achieving higher accuracy and efficiency over previous methods like AthenaTest.

Contribution

The paper introduces A3Test, a novel assertion-augmented test case generation method that leverages domain adaptation and verification to enhance correctness and speed.

Findings

Achieves 147% more correct test cases than AthenaTest.

Improves method coverage by 15%.

Operates at 97.2% faster speed.

Abstract

Test case generation is an important activity, yet a time-consuming and laborious task. Recently, AthenaTest -- a deep learning approach for generating unit test cases -- is proposed. However, AthenaTest can generate less than one-fifth of the test cases correctly, due to a lack of assertion knowledge and test signature verification. In this paper, we propose A3Test, a DL-based test case generation approach that is augmented by assertion knowledge with a mechanism to verify naming consistency and test signatures. A3Test leverages the domain adaptation principles where the goal is to adapt the existing knowledge from an assertion generation task to the test case generation task. We also introduce a verification approach to verify naming consistency and test signatures. Through an evaluation of 5,278 focal methods from the Defects4j dataset, we find that our A3Test (1) achieves 147% more correct test cases and 15% more method coverage, with a lower number of generated test cases than AthenaTest; (2) still outperforms the existing pre-trained models for the test case generation task; (3) contributes substantially to performance improvement via our own proposed assertion pre-training and the verification components; (4) is 97.2% much faster while being more accurate than AthenaTest.