Test Case Recommendations with Distributed Representation of Code Syntactic Features

TL;DR

This paper presents an automated neural network-based method that uses structural and semantic code features to recommend relevant unit tests, reducing developer effort in maintaining test suites amid frequent code changes.

Contribution

It introduces a novel approach leveraging distributed code representations to recommend test cases, improving automation and relevance over prior manual or less integrated methods.

Findings

The approach effectively retrieves similar test cases based on code embeddings.

Recommended test cases reduce developer effort in test maintenance.

The method demonstrates promising results on the Methods2Test dataset.

Abstract

Frequent modifications of unit test cases are inevitable due to software's continuous underlying changes in source code, design, and requirements. Since manually maintaining software test suites is tedious, timely, and costly, automating the process of generation and maintenance of test units will significantly impact the effectiveness and efficiency of software testing processes. To this end, we propose an automated approach which exploits both structural and semantic properties of source code methods and test cases to recommend the most relevant and useful unit tests to the developers. The proposed approach initially trains a neural network to transform method-level source code, as well as unit tests, into distributed representations (embedded vectors) while preserving the importance of the structure in the code. Retrieving the semantic and structural properties of a given method, the approach computes cosine similarity between the method's embedding and the previously-embedded training instances. Further, according to the similarity scores between the embedding vectors, the model identifies the closest methods of embedding and the associated unit tests as the most similar recommendations. The results on the Methods2Test dataset showed that, while there is no guarantee to have similar relevant test cases for the group of similar methods, the proposed approach extracts the most similar existing test cases for a given method in the dataset, and evaluations show that recommended test cases decrease the developers' effort to generating expected test cases.