Improving Compiler Bug Isolation by Leveraging Large Language Models

TL;DR

This paper introduces AutoCBI, a novel bug localization method leveraging large language models to improve compiler bug isolation by summarizing functions and guiding suspicious file reordering, outperforming existing approaches.

Contribution

The paper presents AutoCBI, the first approach to use LLMs for compiler bug localization, enhancing accuracy and efficiency over prior methods.

Findings

AutoCBI isolates significantly more bugs in top ranks than state-of-the-art methods.

Using LLMs improves bug localization effectiveness in large compiler codebases.

Component ablation confirms each part's importance in AutoCBI's success.

Abstract

Compilers play a foundational role in building reliable software systems, and bugs within them can lead to catastrophic consequences. The compilation process typically involves hundreds of files, making traditional automated bug isolation techniques inapplicable due to scalability or effectiveness issues. Current mainstream compiler bug localization techniques have limitations in test program mutation and resource consumption. Inspired by the recent advances of pre-trained Large Language Models (LLMs), we propose an innovative approach named AutoCBI, which (1) uses LLMs to summarize compiler file functions and (2) employs specialized prompts to guide LLM in reordering suspicious file rankings. This approach leverages four types of information: the failing test program, source file function summaries, lists of suspicious files identified through analyzing test coverage, as well as compilation configurations with related output messages, resulting in a refined ranking of suspicious files. Our evaluation of AutoCBI against state-of-the-art approaches (DiWi, RecBi and FuseFL) on 120 real-world bugs from the widely-used GCC and LLVM compilers demonstrates its effectiveness. Specifically, AutoCBI isolates 66.67%/69.23%, 300%/340%, and 100%/57.14% more bugs than RecBi, DiWi, and FuseFL, respectively, in the Top-1 ranked results for GCC/LLVM. Additionally, the ablation study underscores the significance of each component in our approach.