ConCovUp: Effective Agent-Based Test Driver Generation for Concurrency Testing

TL;DR

ConCovUp is a multi-agent framework that combines static analysis and large language models to generate effective concurrent tests, significantly improving shared memory access coverage in multi-threaded programs.

Contribution

It introduces a novel LLM-driven backward tracing method grounded in static analysis for automated concurrent test generation.

Findings

Increases shared memory access pair coverage from 36.6% to 68.1%.

Effectively triggers hard-to-reach shared memory accesses.

Leverages LLM semantic reasoning for complex path constraint deduction.

Abstract

Concurrency testing is essential to improve the reliability and security of multi-threaded programs. Dynamic analysis tools, such as TSan, depend on high-quality test drivers that reach critical shared-memory interactions at runtime. However, current testing practices predominantly focus on sequential logic, leaving a gap in automated concurrent test generation. Recently, large language models (LLMs) have shown promise in generating sequential tests, but they struggle to produce effective concurrent tests without a deep understanding of concurrency semantics. This paper presents ConCovUp, a multi-agent framework that combines LLMs with program analysis. ConCovUp grounds test generation in static analysis to extract shared memory accesses and their calling contexts. To trigger hard-to-reach accesses, it introduces an LLM-driven backward tracing approach, leveraging the model's semantic reasoning to deduce concrete inputs that satisfy complex path constraints, and iteratively refines the generated tests via dynamic execution feedback. Our evaluation on nine real-world C/C++ libraries shows that ConCovUp improves average Shared Memory Access Pair Coverage (SMAP Coverage) from 36.6% to 68.1% over the general Claude Code agent baseline.