DeepRTL: Bridging Verilog Understanding and Generation with a Unified Representation Model

TL;DR

DeepRTL is a unified model based on CodeT5+ that improves both understanding and generation of Verilog code from natural language, addressing previous weak alignment issues and introducing new benchmarks and evaluation metrics.

Contribution

The paper introduces DeepRTL, the first unified model for Verilog understanding and generation, along with a new benchmark and semantic evaluation metrics.

Findings

DeepRTL outperforms GPT-4 in Verilog understanding tasks.

DeepRTL achieves comparable performance to OpenAI's o1-preview in Verilog generation.

New benchmarks and metrics better capture semantic understanding of Verilog code.

Abstract

Recent advancements in large language models (LLMs) have shown significant potential for automating hardware description language (HDL) code generation from high-level natural language instructions. While fine-tuning has improved LLMs' performance in hardware design tasks, prior efforts have largely focused on Verilog generation, overlooking the equally critical task of Verilog understanding. Furthermore, existing models suffer from weak alignment between natural language descriptions and Verilog code, hindering the generation of high-quality, synthesizable designs. To address these issues, we present DeepRTL, a unified representation model that excels in both Verilog understanding and generation. Based on CodeT5+, DeepRTL is fine-tuned on a comprehensive dataset that aligns Verilog code with rich, multi-level natural language descriptions. We also introduce the first benchmark for Verilog understanding and take the initiative to apply embedding similarity and GPT Score to evaluate the models' understanding capabilities. These metrics capture semantic similarity more accurately than traditional methods like BLEU and ROUGE, which are limited to surface-level n-gram overlaps. By adapting curriculum learning to train DeepRTL, we enable it to significantly outperform GPT-4 in Verilog understanding tasks, while achieving performance on par with OpenAI's o1-preview model in Verilog generation tasks.