Understanding Inference-Time Token Allocation and Coverage Limits in Agentic Hardware Verification

TL;DR

This paper empirically analyzes the limits of LLM-based hardware verification, focusing on token allocation, coverage gaps, and efficiency improvements through domain specialization.

Contribution

It introduces a two-tier agentic framework that characterizes coverage holes, tracks token usage, and demonstrates efficiency gains with domain-specific models.

Findings

Enhanced system achieves 95-99% coverage with 4-13x fewer tokens.

Domain specialization shifts token allocation toward reasoning.

The study exposes fundamental limits of purely LLM-driven verification.

Abstract

Coverage closure is the most time-consuming phase of hardware verification, and recent large language model (LLM)-based coding agents offer a promising approach to automated stimulus generation. However, prior LLM-based flows do not systematically analyze which coverage holes remain difficult to close or how inference-time computation is allocated during agentic verification. As a result, the efficiency limits and failure modes of LLM-based coverage closure remain poorly understood, particularly for large designs. We present an empirical study using a two-tier agentic framework comprising a base Codex agent and an enhanced domain-specialized LangGraph system. Our framework enables a taxonomy of coverage holes: methodology-bound ceilings (integration tied-off hardware, infeasible boundaries, dead code) and reasoning frontiers (protocol sequencing, multi-module pipeline warm-up, narrow timing conditions), exposing fundamental limits of purely LLM-driven approaches. We further instrument the system to track token usage across six categories, including system prompt, design comprehension, stimulus generation, coverage feedback, error recovery, and agentic overhead. We show that domain specialization shifts token allocation toward coverage-directed reasoning and improves efficiency. Across designs, the enhanced system achieves comparable or higher coverage (95-99%) while using 4-13x fewer tokens and converging to coverage targets 2-4x faster than a general-purpose baseline. Our results characterize the limits of LLM-based coverage closure, inform benchmark design and human escalation strategies, and guide profile-driven agent design for hardware verification.