From Reaction to Anticipation: Proactive Failure Recovery through Agentic Task Graph for Robotic Manipulation

TL;DR

AgentChord is a proactive robotic manipulation framework that models tasks with anticipatory recovery branches, enabling immediate failure responses and improving success rates in complex environments.

Contribution

It introduces a novel agentic system that anticipates failures by enriching task graphs with recovery strategies, reducing latency and increasing robustness.

Findings

Significantly improves success rates in manipulation tasks.

Reduces latency by enabling immediate failure recovery.

Enhances robustness and autonomy in real-world robotic systems.

Abstract

Although robotic manipulation has made significant progress, reliable execution remains challenging because task failures are inevitable in dynamic and unstructured environments. To handle such failures, existing frameworks typically follow a stepwise detect-reason-recover pipeline, which often incurs high latency and limited robustness due to delayed reasoning and reactive planning. Inspired by the human capability to anticipate and proactively plan for potential failures, we introduce AgentChord, an agentic system that models a manipulation task as a directed task graph. Before execution, this graph is enriched with anticipatory recovery branches that specify context-aware corrective behaviors, enabling immediate and targeted responses when failures occur. Specifically, AgentChord operates through a choreography of specialized agents: a composer that structures the nominal task graph, an arranger that augments the graph with anticipatory recovery branches, and a conductor that compiles and coordinates executable transitions using low-latency monitors to detect deviations and trigger pre-compiled recoveries without re-planning. Empirical studies on diverse long-horizon bimanual manipulation tasks demonstrate that AgentChord substantially improves success rates and execution efficiency, advancing the reliability and autonomy of real-world robotic systems. The project page is available at: https://shengxu.net/AgentChord/.