Robust Multi-Agent LLMs under Byzantine Faults

TL;DR

This paper introduces Self-Anchored Consensus (SAC), a decentralized protocol for multi-agent LLM systems that resists Byzantine faults and improves reliability without relying on leader-based methods.

Contribution

The paper proposes a novel decentralized filtering protocol, SAC, with robustness guarantees, enhancing multi-agent LLM systems against adversarial Byzantine agents.

Findings

SAC effectively suppresses Byzantine influence in experiments.

SAC improves performance across various communication topologies.

Prior methods degrade under adversarial conditions.

Abstract

Large language model (LLM) agents increasingly collaborate over peer-to-peer networks to improve their reliability. However, these same interactions can also become a source of vulnerability, as unreliable or Byzantine agents may sway neighboring agents toward incorrect conclusions and degrade overall system performance. Existing methods rely on leader-based coordination or self-reported confidence, both of which are susceptible to adversarial manipulation. We study decentralized LLM multi-agent systems (LLM-MAS) and propose Self-Anchored Consensus (SAC), a fully decentralized iterative filter-and-refine protocol in which agents iteratively exchange responses, locally evaluate and filter unreliable messages, and refine their own outputs. We present $(F{+}1)$-robustness conditions for the communication graph that ensure honest agents preserve and propagate reliable information despite Byzantine influence. Experiments on mathematical and commonsense reasoning benchmarks show that SAC effectively suppresses Byzantine influence and consistently improves performance across diverse communication topologies, whereas prior methods degrade under adversarial conditions.