Wit-HW: Bug Localization in Hardware Design Code via Witness Test Case Generation

TL;DR

Wit-HW introduces a novel approach to hardware bug localization by generating witness test cases to analyze execution differences, significantly improving bug localization accuracy over existing methods.

Contribution

This paper presents Wit-HW, an automated framework that transforms bug localization into a test generation problem using witness test cases and spectrum analysis, outperforming prior techniques.

Findings

Locates 49% of bugs at Top-1 rank

Achieves 88% within Top-10 ranks

Effective on real-world hardware bugs

Abstract

Debugging hardware designs requires significant manual effort during hardware development. After engineers identify a bug-triggering test case in simulation-based hardware verification, they usually spend considerable time analyzing the execution trace to localize the bug. Although numerous automated hardware debugging techniques exist, they are not applicable to large designs and deep bugs. A primary reason for their limitations is that these techniques only utilize the information of a single bug-triggering test case for bug localization, which prevents them from effectively analyzing intricate hardware systems and figure out the root cause of bugs. To solve this problem, in this paper, we transform the hardware bug localization problem into a test generation problem, aiming to find a set of effective witness test cases beyond the initial bug-triggering test case to enhance hardware bug localization. Witness test cases refer to the cases that do not trigger the bug in the faulty design. By analyzing the execution differences between passing and failing test cases with spectrum-based method, we can eliminate innocent design statements and localize the buggy ones. To further refine the suspicious area, we define the criteria for effective witness test cases and use a mutation-based strategy to generate such test cases. Based on this approach, we propose an automated hardware bug localization framework named Wit-HW. We evaluate Wit-HW on 41 bugs from various hardware designs. The experimental results show that Wit-HW effectively localize 49%, 73%, 88% bugs within Top-1, Top-5, Top-10 ranks, significantly outperforming state-of-the-art bug localization techniques. Additionally, we evaluate Wit-HW on 13 real-world bugs collected from open-source hardware projects, showcasing the robust performance of our method.