Improving Deep Assertion Generation via Fine-Tuning Retrieval-Augmented Pre-trained Language Models

TL;DR

RetriGen is a retrieval-augmented deep assertion generation method that combines lexical and semantic retrieval with a pre-trained language model to improve automatic unit test assertion generation accuracy.

Contribution

This paper introduces RetriGen, a novel hybrid retrieval and PLM-based approach that enhances assertion generation by integrating lexical and semantic retrieval with deep learning.

Findings

RetriGen outperforms six state-of-the-art methods in accuracy and CodeBLEU.

It achieves 57.66% assertion accuracy, a 50.66% improvement over baselines.

RetriGen demonstrates significant effectiveness across large-scale datasets.

Abstract

Unit testing validates the correctness of the units of the software system under test and serves as the cornerstone in improving software quality and reliability. To reduce manual efforts in writing unit tests, some techniques have been proposed to automatically generate test assertions, with recent integration-based approaches considered state-of-the-art. Despite being promising, such integration-based approaches face several limitations, including reliance on lexical matching for assertion retrieval and a limited training corpus for assertion generation. This paper proposes a novel retrieval-augmented deep assertion generation approach, namely RetriGen, based on a hybrid retriever and a pre-trained language model (PLM)-based generator. Given a focal-test, RetriGen first builds a hybrid assertion retriever to search for the most relevant Test-Assert Pair from external codebases. The retrieval process considers lexical similarity and semantical similarity via a token-based and an embedding-based retriever, respectively. RetriGen then treats assertion generation as a sequence-to-sequence task and designs a PLM-based assertion generator to predict a correct assertion. We conduct extensive experiments to evaluate RetriGen against six state-of-the-art approaches across two large-scale datasets and two metrics. The results demonstrate that RetriGen achieves 57.66% accuracy and 73.24% CodeBLEU, outperforming all baselines with average improvements of 50.66% and 14.14%, respectively.