Towards LLM-based Root Cause Analysis of Hardware Design Failures

TL;DR

This paper investigates how large language models can be used to identify root causes of hardware design failures, demonstrating high accuracy in diagnosing bugs during synthesis and simulation processes.

Contribution

It introduces a novel application of LLMs for hardware root cause analysis, showing their effectiveness in diagnosing design issues with high accuracy.

Findings

OpenAI's o3-mini model achieved 100% accuracy in correct root cause determination.

Retrieval-augmented generation improved performance to over 90%.

Models outperformed traditional methods in bug diagnosis accuracy.

Abstract

With advances in large language models (LLMs), new opportunities have emerged to develop tools that support the digital hardware design process. In this work, we explore how LLMs can assist with explaining the root cause of design issues and bugs that are revealed during synthesis and simulation, a necessary milestone on the pathway towards widespread use of LLMs in the hardware design process and for hardware security analysis. We find promising results: for our corpus of 34 different buggy scenarios, OpenAI's o3-mini reasoning model reached a correct determination 100% of the time under pass@5 scoring, with other state of the art models and configurations usually achieving more than 80% performance and more than 90% when assisted with retrieval-augmented generation.