LANCET: Neural Intervention via Structural Entropy for Mitigating Faithfulness Hallucinations in LLMs

TL;DR

Lancet introduces a structural entropy-based neural intervention framework to precisely identify and block pathways causing hallucinations in large language models, significantly improving faithfulness without impairing overall performance.

Contribution

The paper presents Lancet, a novel method that leverages structural entropy and gradient analysis for targeted neural intervention to reduce hallucinations in LLMs.

Findings

Lancet outperforms existing methods on hallucination benchmarks.

The intervention preserves the general capabilities of the model.

Structural entropy effectively identifies propagation pathways of hallucinations.

Abstract

Large Language Models have revolutionized information processing, yet their reliability is severely compromised by faithfulness hallucinations. While current approaches attempt to mitigate this issue through node-level adjustments or coarse suppression, they often overlook the distributed nature of neural information, leading to imprecise interventions. Recognizing that hallucinations propagate through specific forward transmission pathways like an infection, we aim to surgically block this flow using precise structural analysis. To leverage this, we propose Lancet, a novel framework that achieves precise neural intervention by leveraging structural entropy and hallucination difference ratios. Lancet first locates hallucination-prone neurons via gradient-driven contrastive analysis, then maps their propagation pathways by minimizing structural entropy, and finally implements a hierarchical intervention strategy that preserves general model capabilities. Comprehensive evaluations across hallucination benchmark datasets demonstrate that Lancet significantly outperforms state-of-the-art methods, validating the effectiveness of our surgical approach to neural intervention.