Spec2Cov: An Agentic Framework for Code Coverage Closure of Digital Hardware Designs

TL;DR

Spec2Cov is an innovative agentic framework that leverages large language models to automate and accelerate the process of coverage closure in hardware verification, reducing manual effort and improving efficiency.

Contribution

The paper introduces Spec2Cov, a novel framework that autonomously generates test stimuli from specifications using LLMs, coordinating with hardware simulators for coverage closure.

Findings

Achieves 100% coverage on simple designs

Reaches up to 49% coverage on complex designs

Demonstrates effectiveness across 26 diverse hardware designs

Abstract

Hardware verification is one of the most challenging stages of the hardware design process, requiring significant time and resources to ensure a design is fully validated and production-ready. Verification teams aim to maximize design coverage while ensuring correct behavior and alignment with the specification. Coverage closure, which relies on iterative constrained-random and directed testing, is still largely manual and therefore slow and labor-intensive. Recent advances show that the code generation capabilities of Large Language Models (LLMs) can be integrated with external tools to build agentic workflows that autonomously perform hardware design and verification tasks. In this work, we introduce Spec2Cov, an agentic framework that automatically and iteratively generates test stimulus directly from design specifications to accelerate coverage closure. Spec2Cov coordinates interactions between an LLM and a hardware simulator, managing compilation and simulation errors, parsing coverage reports, and feeding results back to the model for refinement. We present features that improve Spec2Cov's effectiveness without additional fine-tuning and evaluate their impact. Across 26 designs of varying size and complexity, including problems from the CVDP benchmark suite, Spec2Cov demonstrates promising performance, achieving 100% coverage on simpler designs and up to 49% on more complex designs.