CortexDebate: Debating Sparsely and Equally for Multi-Agent Debate

TL;DR

CortexDebate introduces a sparse, dynamically optimized multi-agent debate framework for LLMs, reducing input overload and overconfidence issues, leading to improved debate effectiveness across diverse tasks.

Contribution

The paper proposes CortexDebate, a novel multi-agent debate method that constructs a sparse debating graph and employs a trust-based optimization module inspired by human brain connectivity.

Findings

Outperforms existing MAD methods on eight datasets.

Effectively mitigates input overload and overconfidence issues.

Demonstrates versatility across four different task types.

Abstract

Nowadays, single Large Language Model (LLM) struggles with critical issues such as hallucination and inadequate reasoning abilities. To mitigate these issues, Multi-Agent Debate (MAD) has emerged as an effective strategy, where LLM agents engage in in-depth debates with others on tasks. However, existing MAD methods face two major issues: (a) too lengthy input contexts, which causes LLM agents to get lost in plenty of input information and experiences performance drop; and (b) the overconfidence dilemma, where self-assured LLM agents dominate the debate, leading to low debating effectiveness. To address these limitations, we propose a novel MAD method called "CortexDebate". Inspired by the human brain's tendency to establish a sparse and dynamically optimized network among cortical areas governed by white matter, CortexDebate constructs a sparse debating graph among LLM agents, where each LLM agent only debates with the ones that are helpful to it. To optimize the graph, we propose a module named McKinsey-based Debate Matter (MDM), which acts as an artificial analog to white matter. By integrating the McKinsey Trust Formula, a well-established measure of trustworthiness from sociology, MDM enables credible evaluations that guide graph optimization. The effectiveness of our CortexDebate has been well demonstrated by extensive experimental results across eight datasets from four task types.