Improving LLM-Driven Test Generation by Learning from Mocking Information

TL;DR

This paper introduces MOCKMILL, an LLM-based test generation method that uses mocking information from existing tests to produce more effective and comprehensive unit tests.

Contribution

It presents a novel approach that leverages developer-defined mocking information to improve the quality and coverage of LLM-generated tests.

Findings

MOCKMILL generates tests that cover more code lines.

It kills more mutants than baseline approaches.

Tests are executable and guided by mocking data.

Abstract

Large Language Models (LLMs) have recently shown strong potential for automated unit test generation. This has motivated us to investigate whether developer-defined test doubles (commonly referred to as mocks) available in existing test suites can be leveraged to improve LLM-driven test generation. To this end, we propose MOCKMILL, an LLM-based technique and tool that generates test cases by exploiting mocking information automatically extracted from developer-written tests. MOCKMILL targets components that are replaced by test doubles in existing tests and uses the encoded stubbings and interaction expectations to guide test generation, combined with an iterative generation-and-repair process to ensure executable tests. We evaluated MOCKMILL on 10 open-source classes from six Java projects using four LLMs, and compared the generated tests with existing project tests and tests produced by baseline approaches. The results show that MOCKMILL's tests cover lines of code and kill mutants that existing tests and baseline-generated tests miss. Overall, our findings provide preliminary evidence that leveraging mocking information is a complementary and effective way to enhance LLM-based test generation.