VERIRL: Boosting the LLM-based Verilog Code Generation via Reinforcement Learning

TL;DR

This paper introduces VERIRL, a reinforcement learning framework for Verilog code generation, leveraging a new dataset and novel reward refinement techniques to achieve state-of-the-art results in hardware description language synthesis.

Contribution

The paper presents a tailored RL approach for Verilog, including a new dataset, a Trace-back Rescore mechanism, and a sample-balanced weighting strategy, advancing hardware code generation.

Findings

Achieved state-of-the-art performance in Verilog code generation

Demonstrated improved test pass rate and functional correctness

Outperformed existing methods like CraftRTL and DeepSeek-style approaches

Abstract

Recent advancements in code generation have shown remarkable success across software domains, yet hardware description languages (HDLs) such as Verilog remain underexplored due to their concurrency semantics, syntactic rigidity, and simulation complexity. In this work, we address these challenges by introducing a reinforcement learning (RL) framework tailored for Verilog code generation. We first construct Veribench-53K, a high-quality dataset curated from over 700K Verilog problems, enriched with structured prompts, complexity labels, and diverse testbenches. To tackle the problem of sparse and noisy reward signals, we propose a Trace-back based Rescore mechanism that leverages reasoning paths and iterative refinement to enhance feedback reliability and support reward model training. Furthermore, to mitigate catastrophic forgetting and overfitting during RL fine-tuning, we introduce a sample-balanced weighting strategy that adaptively balances learning dynamics based on reward-probability distributions. These innovations are integrated into an iterative RL pipeline that co-evolves the policy and reward models. In contrast to recent work such as CraftRTL, which relies on large-scale closed-source model distillation, and DeepSeek-style approaches that struggle with sparse feedback, our method demonstrates superior performance using a smaller but high-quality dataset combined with RL optimization. Experiments on Verilog generation tasks demonstrate state-of-the-art performance, with substantial gains in test pass rate, functional correctness, and compilation robustness. Our findings highlight the potential of RL-driven approaches for structured code generation in hardware-centric domains. VERIRL is publicly available at https://github.com/omniAI-Lab/VeriRL.