DS@GT at Touch\'e: Large Language Models for Retrieval-Augmented Debate

TL;DR

This paper investigates large language models' capabilities in structured debate and evaluation, using retrieval-augmented methods to assess their performance across multiple metrics in a competitive setting.

Contribution

It introduces a novel framework for LLM-based debate and evaluation, applying retrieval-augmented techniques and providing comprehensive benchmarking across multiple models.

Findings

LLMs perform well with relevant arguments in debates

Models tend to be verbose in responses

Evaluation consistency is maintained across models

Abstract

Large Language Models (LLMs) demonstrate strong conversational abilities. In this Working Paper, we study them in the context of debating in two ways: their ability to perform in a structured debate along with a dataset of arguments to use and their ability to evaluate utterances throughout the debate. We deploy six leading publicly available models from three providers for the Retrieval-Augmented Debate and Evaluation. The evaluation is performed by measuring four key metrics: Quality, Quantity, Manner, and Relation. Throughout this task, we found that although LLMs perform well in debates when given related arguments, they tend to be verbose in responses yet consistent in evaluation. The accompanying source code for this paper is located at https://github.com/dsgt-arc/touche-2025-rad.