On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents

TL;DR

This paper evaluates the resilience of large language model-based multi-agent systems against faulty or malicious agents, proposing methods to simulate faults and mechanisms to improve system robustness across various tasks.

Contribution

It introduces AutoTransform and AutoInject to simulate agent faults, and proposes Challenger and Inspector mechanisms to enhance system resilience, with extensive experimental validation.

Findings

Hierarchical system structures are more resilient to faults.

Challenger and Inspector significantly reduce error impacts.

Resilience improvements up to 96.4% in error recovery.

Abstract

Large language model-based multi-agent systems have shown great abilities across various tasks due to the collaboration of expert agents, each focusing on a specific domain. However, the impact of clumsy or even malicious agents--those who frequently make errors in their tasks--on the overall performance of the system remains underexplored. This paper investigates: (1) What is the resilience of various system structures (e.g., A$\rightarrow$B$\rightarrow$C, A$\leftrightarrow$B$\leftrightarrow$C) under faulty agents, on different downstream tasks? (2) How can we increase system resilience to defend against these agents? To simulate faulty agents, we propose two approaches--AutoTransform and AutoInject--which introduce mistakes into the agents' responses. Experiments on four downstream tasks using six systems show that the "hierarchical" structure, i.e., A$\rightarrow$(B$\leftrightarrow$C), exhibits superior resilience with the lowest performance drop of 5.5%, compared to 10.5% and 23.7% of other two structures. To further improve resilience, we introduce (1) Challenger, that introduces a mechanism for each agent to challenge others' outputs, and (2) Inspector, an additional agent to review and correct messages, recovering up to 96.4% errors made by faulty agents. Our code and data are available at https://github.com/CUHK-ARISE/MAS-Resilience.