SelfHeal: Empirical Fix Pattern Analysis and Bug Repair in LLM Agents

TL;DR

This paper presents an empirical study of bug fix patterns in LLM agents, introduces a new bug dataset, and proposes SelfHeal, a multi-agent system that effectively repairs bugs in LLM agents.

Contribution

It is the first to analyze bug fix patterns in LLM agents, create a benchmark dataset, and develop a multi-agent repair system that outperforms existing methods.

Findings

SelfHeal significantly outperforms baseline approaches.

The dataset contains 37 runtime buggy instances with fixes.

Bug fix patterns vary across platforms and languages.

Abstract

Large Language Models (LLMs) have transformed software development and AI applications. While LLMs are designed for text processing, LLM agents extend this capability by enabling autonomous actions, tool use, and multi-step task completion. As this field grows, developers face new challenges in debugging these complex systems. To address this challenge, we present the first empirical study on bug fix patterns in LLM agents. We study buggy posts and code snippets from three platforms: Stack Overflow, GitHub, and HuggingFace Forums. We examine their fix patterns, the components where fixes are applied, and the programming languages and frameworks involved. Furthermore, we introduce AgentDefect, the first benchmark dataset for bugs in LLM agents. The dataset contains 37 runtime buggy instances along with fixed code and test files. Finally, we present SelfHeal, a multi-agent system designed to fix bugs in LLM agents. The system leverages two independent ReAct agents: the fix agent and the critic agent. These agents use tools that provide both internal knowledge (fix rules) and external knowledge (web search) to propose and validate fixes. Our evaluation shows that SelfHeal with Gemini 3 Pro as the backbone LLM outperforms both baseline and state-of-the-art approaches by a significant margin.