Large Language Model for Verilog Generation with Code-Structure-Guided Reinforcement Learning

TL;DR

This paper presents VeriSeek, a reinforcement learning-enhanced large language model that effectively generates Verilog code by leveraging code structure feedback, overcoming data scarcity and capturing parallel code structures.

Contribution

Introduces VeriSeek, a novel reinforcement learning approach that improves Verilog code generation by utilizing code structure information, addressing data limitations and structural differences from software code.

Findings

VeriSeek outperforms existing methods on multiple benchmarks.

Reinforcement learning with code structure feedback enhances Verilog code quality.

The approach effectively captures parallel code structures in Verilog.

Abstract

Recent advancements in large language models (LLMs) have sparked significant interest in the automatic generation of Register Transfer Level (RTL) designs, particularly using Verilog. Current research on this topic primarily focuses on pre-training and instruction tuning, but the effectiveness of these methods is constrained by the limited availability of training data, as public Verilog code is far less abundant than software code. In particular, these methods struggle to effectively capture Verilog parallel code structures, which fundamentally differ from the imperative, sequential control flow typical in most software programming languages. This paper introduces VeriSeek, an LLM enhanced by reinforcement learning using a limited amount of high-quality training data to achieve high Verilog code generation performance. Our reinforcement learning approach employs code structure information as feedback signals to refine the pre-trained model, enabling it to effectively learn important patterns from Verilog code with parallel structures. Experiments show that VeriSeek outperforms state-of-the-art methods across multiple benchmarks.