Argumentative Large Language Models for Explainable and Contestable Claim Verification

TL;DR

This paper introduces Argumentative LLMs (ArgLLMs), which enhance large language models with argumentative reasoning to improve explainability and contestability in claim verification tasks.

Contribution

The paper proposes a novel method to augment LLMs with argumentation frameworks, enabling formal reasoning for better explainability and contestability.

Findings

ArgLLMs outperform state-of-the-art techniques in claim verification.

ArgLLMs provide interpretable decision explanations.

Formal properties of contestability are defined and evaluated.

Abstract

The profusion of knowledge encoded in large language models (LLMs) and their ability to apply this knowledge zero-shot in a range of settings makes them promising candidates for use in decision-making. However, they are currently limited by their inability to provide outputs which can be faithfully explained and effectively contested to correct mistakes. In this paper, we attempt to reconcile these strengths and weaknesses by introducing \emph{argumentative LLMs (ArgLLMs)}, a method for augmenting LLMs with argumentative reasoning. Concretely, ArgLLMs construct argumentation frameworks, which then serve as the basis for formal reasoning in support of decision-making. The interpretable nature of these argumentation frameworks and formal reasoning means that any decision made by ArgLLMs may be explained and contested. We evaluate ArgLLMs' performance experimentally in comparison with state-of-the-art techniques, in the context of the decision-making task of claim verification. We also define novel properties to characterise contestability and assess ArgLLMs formally in terms of these properties.