LSPRAG: LSP-Guided RAG for Language-Agnostic Real-Time Unit Test Generation

TL;DR

LSPRAG introduces a language-agnostic, real-time unit test generation framework leveraging LSP back-ends to provide precise context to LLMs, significantly improving code coverage across multiple programming languages.

Contribution

LSPRAG is the first framework to use LSP back-ends for real-time, language-agnostic context retrieval in unit test generation, reducing engineering effort.

Findings

Increased line coverage by up to 174.55% for Golang

Achieved up to 213.31% coverage improvement for Java

Improved coverage by 31.57% for Python

Abstract

Automated unit test generation is essential for robust software development, yet existing approaches struggle to generalize across multiple programming languages and operate within real-time development. While Large Language Models (LLMs) offer a promising solution, their ability to generate high coverage test code depends on prompting a concise context of the focal method. Current solutions, such as Retrieval-Augmented Generation, either rely on imprecise similarity-based searches or demand the creation of costly, language-specific static analysis pipelines. To address this gap, we present LSPRAG, a framework for concise-context retrieval tailored for real-time, language-agnostic unit test generation. LSPRAG leverages off-the-shelf Language Server Protocol (LSP) back-ends to supply LLMs with precise symbol definitions and references in real time. By reusing mature LSP servers, LSPRAG provides an LLM with language-aware context retrieval, requiring minimal per-language engineering effort. We evaluated LSPRAG on open-source projects spanning Java, Go, and Python. Compared to the best performance of baselines, LSPRAG increased line coverage by up to 174.55% for Golang, 213.31% for Java, and 31.57% for Python.