DUET: Agentic Design Understanding via Experimentation and Testing

TL;DR

DUET is a methodology that enhances AI understanding of complex hardware designs by iterative experimentation, testing, and hypothesis integration, leading to improved formal verification performance.

Contribution

It introduces a novel, iterative approach mimicking hardware experts' methods to improve AI comprehension of RTL code using experimentation and testing.

Findings

DUET improves AI performance on formal verification tasks.

Iterative testing enhances understanding of complex RTL designs.

Method bridges gap between AI inference and hardware design complexity.

Abstract

AI agents powered by large language models (LLMs) are being used to solve increasingly complex software engineering challenges, but struggle with hardware design tasks. Register Transfer Level (RTL) code presents a unique challenge for LLMs, as it encodes complex, dynamic, time-evolving behaviors using the low-level language features of SystemVerilog. LLMs struggle to infer these complex behaviors from the syntax of RTL alone, which limits their ability to complete all downstream tasks like code completion, documentation, or verification. In response to this issue, we present DUET: a general methodology for developing Design Understanding via Experimentation and Testing. DUET mimics how hardware design experts develop an understanding of complex designs: not just via a one-off readthrough of the RTL, but via iterative experimentation using a number of tools. DUET iteratively generates hypotheses, tests them with EDA tools (e.g., simulation, waveform inspection, and formal verification), and integrates the results to build a bottom-up understanding of the design. In our evaluations, we show that DUET improves AI agent performance on formal verification, when compared to a baseline flow without experimentation.