GRPO with State Mutations: Improving LLM-Based Hardware Test Plan Generation

TL;DR

This paper explores enhancing large language models for hardware test plan generation by developing a specialized training method called GRPO with State Mutation, significantly improving their ability to generate valid RTL stimuli.

Contribution

The paper introduces a novel training approach combining supervised fine-tuning with reinforcement learning and state mutations, tailored for LLMs in RTL verification.

Findings

Baseline models achieve only 15.7-21.7% success in generating passing stimuli.

The proposed method improves success rate to 33.3%.

The approach outperforms larger general-purpose models in hardware verification tasks.

Abstract

RTL design often relies heavily on ad-hoc testbench creation early in the design cycle. While large language models (LLMs) show promise for RTL code generation, their ability to reason about hardware specifications and generate targeted test plans remains largely unexplored. We present the first systematic study of LLM reasoning capabilities for RTL verification stimuli generation, establishing a two-stage framework that decomposes test plan generation from testbench execution. Our benchmark reveals that state-of-the-art models, including DeepSeek-R1 and Claude-4.0-Sonnet, achieve only 15.7-21.7% success rates on generating stimuli that pass golden RTL designs. To improve LLM generated stimuli, we develop a comprehensive training methodology combining supervised fine-tuning with a novel reinforcement learning approach, GRPO with State Mutation (GRPO-SMu), which enhances exploration by varying input mutations. Our approach leverages a tree-based branching mutation strategy to construct training data comprising equivalent and mutated trees, moving beyond linear mutation approaches to provide rich learning signals. Training on this curated dataset, our 7B parameter model achieves a 33.3% golden test pass rate and a 13.9% mutation detection rate, representing a 17.6% absolute improvement over baseline and outperforming much larger general-purpose models. These results demonstrate that specialized training methodologies can significantly enhance LLM reasoning capabilities for hardware verification tasks, establishing a foundation for automated sub-unit testing in semiconductor design workflows.