Heterogeneous Consensus-Progressive Reasoning for Efficient Multi-Agent Debate

TL;DR

HCP-MAD introduces a three-stage adaptive multi-agent debate framework that improves accuracy and reduces token costs by dynamically adjusting collaboration complexity based on task difficulty.

Contribution

The paper proposes HCP-MAD, a novel multi-agent debate method that uses consensus signals for progressive reasoning and adaptive collaboration to handle tasks efficiently.

Findings

HCP-MAD achieves higher accuracy on multiple benchmarks.

It significantly reduces token costs compared to existing methods.

The three-stage mechanism effectively adapts to task complexity.

Abstract

Multi-Agent Debate (MAD) is a collaborative framework in which multiple agents iteratively refine solutions through the generation of reasoning and alternating critique cycles. Current work primarily optimizes intra-round topologies and inter-round interactions separately, which still results in high token costs regardless of task complexity. This work introduces Heterogeneous Consensus-Progressive Reasoning for Efficient Multi-Agent Debate (HCP-MAD), leveraging consensus as a dynamic signal to facilitate progressive reasoning. The core motivation is that a majority of straightforward tasks can be effectively resolved via lightweight pair-agent debates, while complex tasks require expanded collaboration. Consequently, HCP-MAD employs a three-stage progressive reasoning mechanism to develop adaptive solutions across varying task complexities. Firstly, Heterogeneous Consensus Verification conducts rapid consensus verification using a pair of heterogeneous agents for early stopping. Next, the Heterogeneous Pair-Agent Debate applies an adaptive stopping criterion to dynamically terminate mutual critique of recorded reasoning traces. Finally, the unresolved tasks are addressed through Escalated Collective Voting by aggregating diverse perspectives from additional agents. Experiments across multiple benchmarks show that HCP-MAD significantly enhances accuracy while substantially reducing token costs.