EmbC-Test: How to Speed Up Embedded Software Testing Using LLMs and RAG

TL;DR

This paper introduces EmbC-Test, a RAG-based approach leveraging large language models to automate embedded C software testing, significantly reducing manual effort and increasing testing efficiency.

Contribution

It presents a novel RAG pipeline that grounds LLMs in project artifacts to automate embedded software testing with high accuracy and efficiency.

Findings

Generated tests are 100% syntactically correct.

85% of tests pass runtime validation.

Potential to save up to 66% testing time.

Abstract

Manual development of automatic tests for embedded C software is a strenuous and time-consuming task that does not scale well. With the accelerating pace of software release cycles, verification increasingly becomes the bottleneck in the embedded development workflow. This paper presents a Retrieval-Augmented Generation (RAG) pipeline as a solution for partial automation of the verification process. By grounding a large language model in project-specific artifacts, the approach reduces hallucinations and improves project alignment. An industrial evaluation showed that the generated tests are 100 % syntactically correct, with 85 % successfully passing runtime validation. The proposed solution has the potential to save up to 66 % of the testing time compared to manual test writing while generating 270 tests per hour.