STEAM: Simulating the InTeractive BEhavior of ProgrAMmers for Automatic Bug Fixing

TL;DR

This paper introduces STEAM, a stage-wise framework that simulates collaborative programmer interactions across bug reporting, diagnosis, patch generation, and verification stages using LLMs, significantly improving automatic bug fixing.

Contribution

STEAM is the first framework to model the interactive, multi-stage bug fixing process with LLMs, enhancing their collaborative bug resolution capabilities.

Findings

Achieved state-of-the-art bug fixing performance on benchmark datasets.

Effectively models multi-stage bug resolution process.

Utilizes ChatGPT for interactive bug fixing simulation.

Abstract

Bug fixing holds significant importance in software development and maintenance. Recent research has made notable progress in exploring the potential of large language models (LLMs) for automatic bug fixing. However, existing studies often overlook the collaborative nature of bug resolution, treating it as a single-stage process. To overcome this limitation, we introduce a novel stage-wise framework named STEAM in this paper. The objective of STEAM is to simulate the interactive behavior of multiple programmers involved in various stages across the bug's life cycle. Taking inspiration from bug management practices, we decompose the bug fixing task into four distinct stages: bug reporting, bug diagnosis, patch generation, and patch verification. These stages are performed interactively by LLMs, aiming to imitate the collaborative abilities of programmers during the resolution of software bugs. By harnessing the collective contribution, STEAM effectively enhances the bug-fixing capabilities of LLMs. We implement STEAM by employing the powerful dialogue-based LLM -- ChatGPT. Our evaluation on the widely adopted bug-fixing benchmark demonstrates that STEAM has achieved a new state-of-the-art level of bug-fixing performance.