Argument-Based Consistency in Toxicity Explanations of LLMs

TL;DR

This paper introduces Argument-based Consistency (ArC), a new evaluation framework for assessing the logical coherence of LLMs' toxicity explanations, revealing their reasoning limitations on complex prompts.

Contribution

It proposes a theoretically-grounded, multi-dimensional criterion (ArC) and six metrics to evaluate LLMs' toxicity explanation consistency, addressing limitations of existing methods.

Findings

LLMs generate plausible explanations for simple prompts.

Reasoning about toxicity becomes inconsistent with nuanced prompts.

Code and explanations are open-sourced for future research.

Abstract

The discourse around toxicity and LLMs in NLP largely revolves around detection tasks. This work shifts the focus to evaluating LLMs' reasoning about toxicity - from their explanations that justify a stance - to enhance their trustworthiness in downstream tasks. Despite extensive research on explainability, it is not straightforward to adopt existing methods to evaluate free-form toxicity explanation due to their over-reliance on input text perturbations, among other challenges. To account for these, we propose a novel, theoretically-grounded multi-dimensional criterion, Argument-based Consistency (ArC), that measures the extent to which LLMs' free-form toxicity explanations reflect an ideal and logical argumentation process. Based on uncertainty quantification, we develop six metrics for ArC to comprehensively evaluate the (in)consistencies in LLMs' toxicity explanations. We conduct several experiments on three Llama models (of size up to 70B) and an 8B Ministral model on five diverse toxicity datasets. Our results show that while LLMs generate plausible explanations to simple prompts, their reasoning about toxicity breaks down when prompted about the nuanced relations between the complete set of reasons, the individual reasons, and their toxicity stances, resulting in inconsistent and irrelevant responses. We open-source our code (https://github.com/uofthcdslab/ArC) and LLM-generated explanations (https://huggingface.co/collections/uofthcdslab/arc) for future works.