Enhancing LLM-Based Test Generation by Eliminating Covered Code

TL;DR

This paper introduces a scalable LLM-based method for automated unit test generation that improves coverage of complex code by using context retrieval and iterative code elimination, outperforming existing approaches.

Contribution

The paper presents a novel approach combining context retrieval and iterative code elimination to enhance LLM-based test generation for complex methods.

Findings

Outperforms state-of-the-art LLM-based methods in coverage.

Effective on complex, real-world code snippets.

Reduces issues related to token limits and reasoning complexity.

Abstract

Automated test generation is essential for software quality assurance, with coverage rate serving as a key metric to ensure thorough testing. Recent advancements in Large Language Models (LLMs) have shown promise in improving test generation, particularly in achieving higher coverage. However, while existing LLM-based test generation solutions perform well on small, isolated code snippets, they struggle when applied to complex methods under test. To address these issues, we propose a scalable LLM-based unit test generation method. Our approach consists of two key steps. The first step is context information retrieval, which uses both LLMs and static analysis to gather relevant contextual information associated with the complex methods under test. The second step, iterative test generation with code elimination, repeatedly generates unit tests for the code slice, tracks the achieved coverage, and selectively removes code segments that have already been covered. This process simplifies the testing task and mitigates issues arising from token limits or reduced reasoning effectiveness associated with excessively long contexts. Through comprehensive evaluations on open-source projects, our approach outperforms state-of-the-art LLM-based and search-based methods, demonstrating its effectiveness in achieving high coverage on complex methods.