Multi-Agent Dialectical Refinement for Enhanced Argument Classification

TL;DR

MAD-ACC introduces a multi-agent dialectical framework that improves argument component classification by leveraging debate-like interactions, enhancing accuracy and transparency without domain-specific training.

Contribution

This work presents MAD-ACC, a novel multi-agent debate system that refines argument classification through dialectical reasoning, outperforming single-agent models and providing explainability.

Findings

Achieved a Macro F1 score of 85.7% on UKP Student Essays corpus.

Outperformed single-agent reasoning baselines significantly.

Provided human-readable debate transcripts for explainability.

Abstract

Argument Mining (AM) is a foundational technology for automated writing evaluation, yet traditional supervised approaches rely heavily on expensive, domain-specific fine-tuning. While Large Language Models (LLMs) offer a training-free alternative, they often struggle with structural ambiguity, failing to distinguish between similar components like Claims and Premises. Furthermore, single-agent self-correction mechanisms often suffer from sycophancy, where the model reinforces its own initial errors rather than critically evaluating them. We introduce MAD-ACC (Multi-Agent Debate for Argument Component Classification), a framework that leverages dialectical refinement to resolve classification uncertainty. MAD-ACC utilizes a Proponent-Opponent-Judge model where agents defend conflicting interpretations of ambiguous text, exposing logical nuances that single-agent models miss. Evaluation on the UKP Student Essays corpus demonstrates that MAD-ACC achieves a Macro F1 score of 85.7%, significantly outperforming single-agent reasoning baselines, without requiring domain-specific training. Additionally, unlike "black-box" classifiers, MAD-ACC's dialectical approach offers a transparent and explainable alternative by generating human-readable debate transcripts that explain the reasoning behind decisions.