FailureMem: A Failure-Aware Multimodal Framework for Autonomous Software Repair

TL;DR

FailureMem is a multimodal framework that enhances automated software repair by integrating structured workflows, localized visual reasoning, and reusable knowledge from past failures, significantly improving repair success rates.

Contribution

It introduces a novel hybrid workflow-agent architecture, region-level visual grounding, and a Failure Memory Bank for reusable repair knowledge, advancing multimodal program repair methods.

Findings

Improves repair success rate by 3.7% over GUIRepair.

Integrates localized visual reasoning with structured workflows.

Transforms failed repairs into reusable knowledge.

Abstract

Multimodal Automated Program Repair (MAPR) extends traditional program repair by requiring models to jointly reason over source code, textual issue descriptions, and visual artifacts such as GUI screenshots. While recent LLM-based repair systems have shown promising results, existing approaches face several limitations: rigid workflow pipelines restrict exploration during debugging, visual reasoning is often performed over full-page screenshots without localized grounding, and failed repair attempts are rarely transformed into reusable knowledge. To address these challenges, we propose FailureMem, a multimodal repair framework that integrates three key mechanisms: a hybrid workflow-agent architecture that balances structured localization with flexible reasoning, active perception tools that enable region-level visual grounding, and a Failure Memory Bank that converts past repair attempts into reusable guidance. Experiments on SWE-bench Multimodal demonstrate FailureMem improves the resolved rate over GUIRepair by 3.7%.