LAUDE: LLM-Assisted Unit Test Generation and Debugging of Hardware DEsigns

TL;DR

LAUDE leverages large language models and semantic understanding to automate unit test generation and debugging for hardware designs, significantly improving efficiency and accuracy in identifying and fixing design bugs.

Contribution

This work introduces LAUDE, a novel framework combining LLMs with design semantics for automated hardware unit test generation and debugging.

Findings

Detected bugs in up to 100% of combinational designs

Debugged up to 93% of combinational designs

Effective across diverse hardware design types

Abstract

Unit tests are critical in the hardware design lifecycle to ensure that component design modules are functionally correct and conform to the specification before they are integrated at the system level. Thus developing unit tests targeting various design features requires deep understanding of the design functionality and creativity. When one or more unit tests expose a design failure, the debugging engineer needs to diagnose, localize, and debug the failure to ensure design correctness, which is often a painstaking and intense process. In this work, we introduce LAUDE, a unified unit-test generation and debugging framework for hardware designs that cross-pollinates the semantic understanding of the design source code with the Chain-of-Thought (CoT) reasoning capabilities of foundational Large-Language Models (LLMs). LAUDE integrates prompt engineering and design execution information to enhance its unit test generation accuracy and code debuggability. We apply LAUDE with closed- and open-source LLMs to a large corpus of buggy hardware design codes derived from the VerilogEval dataset, where generated unit tests detected bugs in up to 100% and 93% of combinational and sequential designs and debugged up to 93% and 84% of combinational and sequential designs, respectively.