Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation

TL;DR

This paper investigates self-contradictions in large language models, presenting evaluation, detection, and mitigation methods that are effective, model-agnostic, and publicly available, significantly reducing hallucinated contradictions in generated text.

Contribution

It introduces a novel prompting-based framework for detecting and mitigating self-contradictions in instruction-tuned LMs, applicable without external knowledge retrieval.

Findings

Self-contradictions occur in 17.7% of ChatGPT sentences.

The detector achieves around 80% F1 score.

The mitigation algorithm effectively reduces contradictions while maintaining fluency.

Abstract

Large language models (large LMs) are susceptible to producing text that contains hallucinated content. An important instance of this problem is self-contradiction, where the LM generates two contradictory sentences within the same context. In this work, we present a comprehensive investigation into self-contradiction for various instruction-tuned LMs, covering evaluation, detection, and mitigation. Our primary evaluation task is open-domain text generation, but we also demonstrate the applicability of our approach to shorter question answering. Our analysis reveals the prevalence of self-contradictions, e.g., in 17.7% of all sentences produced by ChatGPT. We then propose a novel prompting-based framework designed to effectively detect and mitigate self-contradictions. Our detector achieves high accuracy, e.g., around 80% F1 score when prompting ChatGPT. The mitigation algorithm iteratively refines the generated text to remove contradictory information while preserving text fluency and informativeness. Importantly, our entire framework is applicable to black-box LMs and does not require retrieval of external knowledge. Rather, our method complements retrieval-based methods, as a large portion of self-contradictions (e.g., 35.2% for ChatGPT) cannot be verified using online text. Our approach is practically effective and has been released as a push-button tool to benefit the public at https://chatprotect.ai/.