Can formal argumentative reasoning enhance LLMs performances?

TL;DR

This paper explores integrating computational argumentation into LLMs to enhance reasoning and conversational abilities without retraining, showing moderate performance gains in initial experiments.

Contribution

It introduces a novel pipeline, MQArgEng, for incorporating argumentation semantics into LLMs and provides preliminary evidence of its potential benefits.

Findings

Moderate performance improvements observed in initial tests.

Argumentation integration shows promise for enhancing LLM reasoning.

Further research needed to develop a full argumentation engine for LLMs.

Abstract

Recent years witnessed significant performance advancements in deep-learning-driven natural language models, with a strong focus on the development and release of Large Language Models (LLMs). These improvements resulted in better quality AI-generated output but rely on resource-expensive training and upgrading of models. Although different studies have proposed a range of techniques to enhance LLMs without retraining, none have considered computational argumentation as an option. This is a missed opportunity since computational argumentation is an intuitive mechanism that formally captures agents' interactions and the information conflict that may arise during such interplays, and so it seems well-suited for boosting the reasoning and conversational abilities of LLMs in a seamless manner. In this paper, we present a pipeline (MQArgEng) and preliminary study to evaluate the effect of introducing computational argumentation semantics on the performance of LLMs. Our experiment's goal was to provide a proof-of-concept and a feasibility analysis in order to foster (or deter) future research towards a fully-fledged argumentation engine plugin for LLMs. Exploratory results using the MT-Bench indicate that MQArgEng provides a moderate performance gain in most of the examined topical categories and, as such, show promise and warrant further research.