REACCEPT: Automated Co-evolution of Production and Test Code Based on Dynamic Validation and Large Language Models

TL;DR

REACCEPT is a novel approach that uses large language models and dynamic validation to fully automate the co-evolution of production and test code, significantly improving accuracy over existing methods.

Contribution

It introduces REACCEPT, the first method leveraging large language models for complete automation of PT co-evolution, surpassing prior heuristic and machine learning approaches.

Findings

Achieved 60.16% update accuracy on obsolete test code.

Surpassed state-of-the-art method CEPROT by 90%.

Effectively aids developers in test code maintenance.

Abstract

Synchronizing production and test code, known as PT co-evolution, is critical for software quality in the software development lifecycle. Existing methods for automatic PT co-evolution either utilize predefined heuristic rules or rely on simple application of machine learning techniques. Due to the limitations of underlying techniques, existing methods either only partially automate PT co-evolution (e.g., only automate obsolete test code identification) or result in low accuracy. In this paper, we propose REACCEPT, a novel approach that leverages large language models and dynamic validation to fully automate PT co-evolution (i.e., capable of both identifying and updating obsolete test cases). REACCEPT relies on experience-based prompt template generation, dynamic validation, and retrieval-augmented generation techniques to accomplish automated PT co-evolution. To evaluate REACCEPT's effectiveness, we extensive experiments with a dataset of 537 Java projects and compared REACCEPT's performance with several state-of-the-art methods. Results show that REACCEPT achieved an update accuracy of 60.16% on correctly identified obsolete test code, surpassing the state-of-the-art technique CEPROT by 90%. This confirms that REACCEPT can effectively assist developers in maintaining test code, improving overall software quality and reducing maintenance effort.