VeriDispatcher: Multi-Model Dispatching through Pre-Inference Difficulty Prediction for RTL Generation Optimization

TL;DR

VeriDispatcher is a framework that intelligently dispatches RTL generation tasks to suitable LLMs based on predicted difficulty, improving accuracy and reducing costs in hardware design automation.

Contribution

It introduces a multi-LLM dispatching approach using pre-inference difficulty prediction, optimizing RTL generation quality and cost efficiency.

Findings

Up to 18% accuracy improvement on RTLLM

Reduces commercial calls by 40% on RTLLM

Maintains accuracy while reducing usage by 25% on VerilogEval

Abstract

Large Language Models (LLMs) show strong performance in RTL generation, but different models excel on different tasks because of architecture and training differences. Prior work mainly prompts or finetunes a single model. What remains not well studied is how to coordinate multiple different LLMs so they jointly improve RTL quality while also reducing cost, instead of running all models and choosing the best output. We define this as the multi-LLM RTL generation problem. We propose VeriDispatcher, a multi-LLM RTL generation framework that dispatches each RTL task to suitable LLMs based on pre-inference difficulty prediction. For each model, we train a compact classifier over semantic embeddings of task descriptions, using difficulty scores derived from benchmark variants that combine syntax, structural similarity, and functional correctness. At inference, VeriDispatcher uses these predictors to route tasks to a selected subset of LLMs. Across 10 diverse LLMs on RTLLM and VerilogEval, VeriDispatcher achieves up to 18% accuracy improvement on RTLLM using only 40% of commercial calls, and on VerilogEval maintains accuracy while reducing commercial usage by 25%, enabling cost-effective, high-quality LLM deployment in hardware design automation.