ArchXBench: A Complex Digital Systems Benchmark Suite for LLM Driven RTL Synthesis
Suresh Purini, Siddhant Garg, Mudit Gaur, Sankalp Bhat, Sohan Mupparapu, Arun Ravindran

TL;DR
ArchXBench is a comprehensive benchmark suite for evaluating LLMs' ability to generate complex RTL digital systems, revealing current limitations of state-of-the-art models in handling advanced design tasks.
Contribution
The paper introduces ArchXBench, a new multi-level benchmark suite with complex digital circuits, and evaluates LLMs' performance, highlighting gaps in their capabilities for RTL synthesis.
Findings
o4-mini-high solves 16 out of 30 benchmarks
Models fail on levels 4 and above
Current LLMs have limited ability for complex RTL design
Abstract
Modern SoC datapaths include deeply pipelined, domain-specific accelerators, but their RTL implementation and verification are still mostly done by hand. While large language models (LLMs) exhibit advanced code-generation abilities for programming languages like Python, their application to Verilog-like RTL remains in its nascent stage. This is reflected in the simple arithmetic and control circuits currently used to evaluate generative capabilities in existing benchmarks. In this paper, we introduce ArchXBench, a six-level benchmark suite that encompasses complex arithmetic circuits and other advanced digital subsystems drawn from domains such as cryptography, image processing, machine learning, and signal processing. Architecturally, some of these designs are purely combinational, others are multi-cycle or pipelined, and many require hierarchical composition of modules. For each…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEmbedded Systems Design Techniques · Parallel Computing and Optimization Techniques · Simulation Techniques and Applications
