Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models

TL;DR

This paper introduces COFT, a coarse-to-fine highlighting method that reduces hallucination in large language models by focusing on key texts at different granularities, improving factual accuracy.

Contribution

The paper proposes a novel COFT approach with recaller, scorer, and selector components to effectively highlight important context, reducing hallucination in retrieval-augmented language models.

Findings

COFT improves F1 score by over 30% on hallucination benchmarks.

COFT enhances performance across long-form tasks like reading comprehension.

The method effectively filters relevant information, reducing factual errors.

Abstract

Generation of plausible but incorrect factual information, often termed hallucination, has attracted significant research interest. Retrieval-augmented language model (RALM) -- which enhances models with up-to-date knowledge -- emerges as a promising method to reduce hallucination. However, existing RALMs may instead exacerbate hallucination when retrieving lengthy contexts. To address this challenge, we propose COFT, a novel \textbf{CO}arse-to-\textbf{F}ine highligh\textbf{T}ing method to focus on different granularity-level key texts, thereby avoiding getting lost in lengthy contexts. Specifically, COFT consists of three components: \textit{recaller}, \textit{scorer}, and \textit{selector}. First, \textit{recaller} applies a knowledge graph to extract potential key entities in a given context. Second, \textit{scorer} measures the importance of each entity by calculating its contextual weight. Finally, \textit{selector} selects high contextual weight entities with a dynamic threshold algorithm and highlights the corresponding paragraphs, sentences, or words in a coarse-to-fine manner. Extensive experiments on the knowledge hallucination benchmark demonstrate the effectiveness of COFT, leading to a superior performance over $30\%$ in the F1 score metric. Moreover, COFT also exhibits remarkable versatility across various long-form tasks, such as reading comprehension and question answering.