An LLM-based Readability Measurement for Unit Tests' Context-aware Inputs

TL;DR

This paper introduces C3, a tool leveraging Large Language Models to measure and improve the readability of unit test inputs by considering the context provided by source code, enhancing automated test generation.

Contribution

The paper proposes C3, a novel context-aware readability measurement tool using LLMs, and integrates it into EvoSuite to generate more readable test inputs based on source code contexts.

Findings

C3 achieves high precision, recall, and F1-score in extracting readability contexts.

EvoSuiteC3 significantly outperforms traditional tools in generating readable string inputs.

Traditional tools like EvoSuite and Randoop produce only 8% readable string inputs.

Abstract

Automated test techniques usually generate unit tests with higher code coverage than manual tests. However, the readability of automated tests is crucial for code comprehension and maintenance. The readability of unit tests involves many aspects. In this paper, we focus on test inputs. The central limitation of existing studies on input readability is that they focus on test codes alone without taking the tested source codes into consideration, making them either ignore different source codes' different readability requirements or require manual efforts to write readable inputs. However, we observe that the source codes specify the contexts that test inputs must satisfy. Based on such observation, we introduce the \underline{C}ontext \underline{C}onsistency \underline{C}riterion (a.k.a, C3), which is a readability measurement tool that leverages Large Language Models to extract primitive-type (including string-type) parameters' readability contexts from the source codes and checks whether test inputs are consistent with those contexts. We have also proposed EvoSuiteC3. It leverages C3's extracted contexts to help EvoSuite generate readable test inputs. We have evaluated C3's performance on $409$ \java{} classes and compared manual and automated tests' readability under C3 measurement. The results are two-fold. First, The Precision, Recall, and F1-Score of C3's mined readability contexts are \precision{}, \recall{}, and \fone{}, respectively. Second, under C3's measurement, the string-type input readability scores of EvoSuiteC3, ChatUniTest (an LLM-based test generation tool), manual tests, and two traditional tools (EvoSuite and Randoop) are $90\%$, $83\%$, $68\%$, $8\%$, and $8\%$, showing the traditional tools' inability in generating readable string-type inputs.