NL-Debugging: Exploiting Natural Language as an Intermediate Representation for Code Debugging

TL;DR

NL-DEBUGGING introduces a novel approach that uses natural language as an intermediate step to enhance code debugging, enabling more effective error understanding and correction by leveraging natural language reasoning.

Contribution

The paper presents NL-DEBUGGING, a framework that employs natural language as an intermediate representation, improving debugging effectiveness and flexibility over traditional code-level methods.

Findings

NL-DEBUGGING outperforms traditional debugging methods.

Natural language reasoning broadens modification possibilities.

The approach effectively addresses complex programming errors.

Abstract

Debugging is a critical aspect of LLM's coding ability. Early debugging efforts primarily focused on code-level analysis, which often falls short when addressing complex programming errors that require a deeper understanding of algorithmic logic. Recent advancements in large language models (LLMs) have shifted attention toward leveraging natural language reasoning to enhance code-related tasks. However, two fundamental questions remain unanswered: What type of natural language format is most effective for debugging tasks? And what specific benefits does natural language reasoning bring to the debugging process? In this paper, we introduce NL-DEBUGGING, a novel framework that employs natural language as an intermediate representation to improve code debugging. By debugging at a natural language level, we demonstrate that NL-DEBUGGING outperforms traditional debugging methods and enables a broader modification space through direct refinement guided by execution feedback. Our findings highlight the potential of natural language reasoning to advance automated code debugging and address complex programming challenges.