Rome was Not Built in a Single Step: Hierarchical Prompting for LLM-based Chip Design

TL;DR

This paper introduces hierarchical prompting techniques for large language models to improve the design of complex hardware modules, enabling automation, reducing costs, and achieving the first LLM-designed processor without human feedback.

Contribution

The paper presents novel hierarchical prompting methods and an automation pipeline that enable LLMs to generate complex hardware designs more effectively than flat prompting.

Findings

Hierarchical prompting enables successful complex hardware design with smaller LLMs.

Reduces HDL generation time and LLM costs.

First LLM-designed processor without human feedback.

Abstract

Large Language Models (LLMs) are effective in computer hardware synthesis via hardware description language (HDL) generation. However, LLM-assisted approaches for HDL generation struggle when handling complex tasks. We introduce a suite of hierarchical prompting techniques which facilitate efficient stepwise design methods, and develop a generalizable automation pipeline for the process. To evaluate these techniques, we present a benchmark set of hardware designs which have solutions with or without architectural hierarchy. Using these benchmarks, we compare various open-source and proprietary LLMs, including our own fine-tuned Code Llama-Verilog model. Our hierarchical methods automatically produce successful designs for complex hardware modules that standard flat prompting methods cannot achieve, allowing smaller open-source LLMs to compete with large proprietary models. Hierarchical prompting reduces HDL generation time and yields savings on LLM costs. Our experiments detail which LLMs are capable of which applications, and how to apply hierarchical methods in various modes. We explore case studies of generating complex cores using automatic scripted hierarchical prompts, including the first-ever LLM-designed processor with no human feedback. Tools for the Recurrent Optimization via Machine Editing (ROME) method can be found at https://github.com/ajn313/ROME-LLM