LLM-Aided Testbench Generation and Bug Detection for Finite-State Machines

TL;DR

This paper explores using GPT-3.5 and GPT-4, combined with EDA tool feedback, to generate and refine testbenches for chip RTL testing, improving coverage and bug detection effectiveness.

Contribution

It introduces a novel method of integrating EDA feedback into LLM-based testbench generation for enhanced chip testing and bug detection.

Findings

Improved test coverage through iterative LLM refinement.

Enhanced bug detection accuracy in RTL designs.

Effective use of commercial EDA feedback with LLMs.

Abstract

This work investigates the potential of tailoring Large Language Models (LLMs), specifically GPT3.5 and GPT4, for the domain of chip testing. A key aspect of chip design is functional testing, which relies on testbenches to evaluate the functionality and coverage of Register-Transfer Level (RTL) designs. We aim to enhance testbench generation by incorporating feedback from commercial-grade Electronic Design Automation (EDA) tools into LLMs. Through iterative feedback from these tools, we refine the testbenches to achieve improved test coverage. Our case studies present promising results, demonstrating that this approach can effectively enhance test coverage. By integrating EDA tool feedback, the generated testbenches become more accurate in identifying potential issues in the RTL design. Furthermore, we extended our study to use this enhanced test coverage framework for detecting bugs in the RTL implementations