Hallucinate Less by Thinking More: Aspect-Based Causal Abstention for Large Language Models

TL;DR

This paper introduces Aspect-Based Causal Abstention (ABCA), a novel method for early abstention in large language models that analyzes internal knowledge diversity through causal inference to reduce hallucinations.

Contribution

ABCA is the first framework to enable early abstention by leveraging causal effects conditioned on knowledge aspects, improving reliability and interpretability.

Findings

ABCA achieves state-of-the-art abstention performance.

ABCA improves the reliability of abstention decisions.

ABCA enhances interpretability of when models choose to abstain.

Abstract

Large Language Models (LLMs) often produce fluent but factually incorrect responses, a phenomenon known as hallucination. Abstention, where the model chooses not to answer and instead outputs phrases such as "I don't know", is a common safeguard. However, existing abstention methods typically rely on post-generation signals, such as generation variations or feedback, which limits their ability to prevent unreliable responses in advance. In this paper, we introduce Aspect-Based Causal Abstention (ABCA), a new framework that enables early abstention by analysing the internal diversity of LLM knowledge through causal inference. This diversity reflects the multifaceted nature of parametric knowledge acquired from various sources, representing diverse aspects such as disciplines, legal contexts, or temporal frames. ABCA estimates causal effects conditioned on these aspects to assess the reliability of knowledge relevant to a given query. Based on these estimates, we enable two types of abstention: Type-1, where aspect effects are inconsistent (knowledge conflict), and Type-2, where aspect effects consistently support abstention (knowledge insufficiency). Experiments on standard benchmarks demonstrate that ABCA improves abstention reliability, achieves state-of-the-art performance, and enhances the interpretability of abstention decisions.