Learning to Debug: LLM-Organized Knowledge Trees for Solving RTL Assertion Failures

TL;DR

GROVE introduces a hierarchical, knowledge-organized framework that enhances LLMs' ability to debug hardware assertion failures by structuring reusable expertise into a dynamic knowledge tree, improving accuracy and efficiency.

Contribution

The paper presents GROVE, a novel hierarchical knowledge management system that organizes debugging expertise into a tree structure, enabling LLMs to more effectively diagnose and fix assertion failures.

Findings

GROVE improves pass@1 and pass@5 metrics on assertion failure cases.

Structured knowledge trees enhance LLM debugging accuracy.

GROVE's iterative zoom effectively retrieves relevant expertise.

Abstract

Debugging is the dominant cost in modern hardware verification, where assertion failures are among the most frequent and expensive to resolve. While Large Language Models (LLMs) show promise, they often fail to capture the precise, reusable expertise that engineers apply, leading to inaccurate responses. We propose GROVE, a hierarchical knowledge management framework that learns and organizes reusable debugging expertise into an LLM-organized knowledge tree for solving assertion failures. GROVE distills debugging knowledge from prior cases and organizes it into a vertical tree of configurable depth, with each node encoding a concise knowledge item and explicit applicability conditions. During training, GROVE uses a parallel, gradient-free loop where an LLM proposes tree modifications as structured JSON edits by learning from the cases. At test time, a budget-aware iterative zoom is performed to navigate the tree, retrieving a small set of applicable knowledge items that guide a base LLM's hypothesis generation and fix proposals. Evaluated on a suite of assertion-failure cases, GROVE delivers consistent gains in pass@1 and pass@5, demonstrating the value of structured knowledge evolution.