Before Humans Join the Team: Diagnosing Coordination Failures in Healthcare Robot Team Simulation

TL;DR

This paper uses LLM-based agent simulations to diagnose coordination failures in healthcare robot teams, aiming to improve safety and trust before human involvement.

Contribution

It introduces a simulation approach with LLM agents to identify coordination issues and analyze how team structure impacts performance in healthcare scenarios.

Findings

Team structure is the main bottleneck for coordination.

Coordination failures are more influenced by hierarchy than contextual knowledge.

Surface-level failures can be diagnosed in simulation to inform safer human integration.

Abstract

As humans move toward collaborating with coordinated robot teams, understanding how these teams coordinate and fail is essential for building trust and ensuring safety. However, exposing human collaborators to coordination failures during early-stage development is costly and risky, particularly in high-stakes domains such as healthcare. We adopt an agent-simulation approach in which all team roles, including the supervisory manager, are instantiated as LLM agents, allowing us to diagnose coordination failures before humans join the team. Using a controllable healthcare scenario, we conduct two studies with different hierarchical configurations to analyze coordination behaviors and failure patterns. Our findings reveal that team structure, rather than contextual knowledge or model capability, constitutes the primary bottleneck for coordination, and expose a tension between reasoning autonomy and system stability. By surfacing these failures in simulation, we prepare the groundwork for safe human integration. These findings inform the design of resilient robot teams with implications for process-level evaluation, transparent coordination protocols, and structured human integration. Supplementary materials, including codes, task agent setup, trace outputs, and annotated examples of coordination failures and reasoning behaviors, are available at: https://byc-sophie.github.io/mas-to-mars/.