Can LLMs Beat Humans in Debating? A Dynamic Multi-agent Framework for Competitive Debate

TL;DR

This paper introduces Agent4Debate, a multi-agent framework based on LLMs that enhances their debating skills, achieving performance comparable to humans through collaborative agents covering research, argumentation, and review stages.

Contribution

The paper presents a novel multi-agent architecture inspired by human debate processes, significantly improving LLMs' competitiveness in structured debates.

Findings

Agent4Debate performs on par with human debaters.

Ablation studies confirm the importance of each agent component.

The framework is evaluated on 66 Chinese debate motions with 200 debates.

Abstract

Competitive debate is a complex task of computational argumentation. Large Language Models (LLMs) suffer from hallucinations and lack competitiveness in this field. To address these challenges, we introduce Agent for Debate (Agent4Debate), a dynamic multi-agent framework based on LLMs designed to enhance their capabilities in competitive debate. Drawing inspiration from human behavior in debate preparation and execution, Agent4Debate employs a collaborative architecture where four specialized agents, involving Searcher, Analyzer, Writer, and Reviewer, dynamically interact and cooperate. These agents work throughout the debate process, covering multiple stages from initial research and argument formulation to rebuttal and summary. To comprehensively evaluate framework performance, we construct the Competitive Debate Arena, comprising 66 carefully selected Chinese debate motions. We recruit ten experienced human debaters and collect records of 200 debates involving Agent4Debate, baseline models, and humans. The evaluation employs the Debatrix automatic scoring system and professional human reviewers based on the established Debatrix-Elo and Human-Elo ranking. Experimental results indicate that the state-of-the-art Agent4Debate exhibits capabilities comparable to those of humans. Furthermore, ablation studies demonstrate the effectiveness of each component in the agent structure.