Auditing Multi-Agent LLM Reasoning Trees Outperforms Majority Vote and LLM-as-Judge

TL;DR

AgentAuditor introduces a reasoning tree-based aggregation method for multi-agent LLM systems, outperforming majority voting and LLM-as-Judge by explicitly analyzing reasoning divergences and conflicts.

Contribution

It presents a novel reasoning tree approach and an optimization method (ACPO) for more accurate multi-agent LLM aggregation, addressing biases and confabulation issues.

Findings

Up to 5% accuracy improvement over majority voting

Up to 3% accuracy improvement over LLM-as-Judge

Effective across 5 different multi-agent settings

Abstract

Multi-agent systems (MAS) can substantially extend the reasoning capacity of large language models (LLMs), yet most frameworks still aggregate agent outputs with majority voting. This heuristic discards the evidential structure of reasoning traces and is brittle under the confabulation consensus, where agents share correlated biases and converge on the same incorrect rationale. We introduce AgentAuditor, which replaces voting with a path search over a Reasoning Tree that explicitly represents agreements and divergences among agent traces. AgentAuditor resolves conflicts by comparing reasoning branches at critical divergence points, turning global adjudication into efficient, localized verification. We further propose Anti-Consensus Preference Optimization (ACPO), which trains the adjudicator on majority-failure cases and rewards evidence-based minority selections over popular errors. AgentAuditor is agnostic to MAS setting, and we find across 5 popular settings that it yields up to 5% absolute accuracy improvement over a majority vote, and up to 3% over using LLM-as-Judge.