Learning Auxiliary Tasks Improves Reference-Free Hallucination Detection in Open-Domain Long-Form Generation

TL;DR

This paper investigates reference-free hallucination detection in open-domain long-form generation, revealing limitations of internal signals and proposing a novel auxiliary task approach, RATE-FT, to improve detection accuracy across models and datasets.

Contribution

It introduces RATE-FT, a new fine-tuning paradigm with an auxiliary task, significantly enhancing hallucination detection in large language models.

Findings

Fine-tuning with auxiliary tasks improves detection accuracy.

Internal model states alone are insufficient for reliable detection.

RATE-FT outperforms standard fine-tuning by +3% on LongFact dataset.

Abstract

Hallucination, the generation of factually incorrect information, remains a significant challenge for large language models (LLMs), especially in open-domain long-form generation. Existing approaches for detecting hallucination in long-form tasks either focus on limited domains or rely heavily on external fact-checking tools, which may not always be available. In this work, we systematically investigate reference-free hallucination detection in open-domain long-form responses. Our findings reveal that internal states (e.g., model's output probability and entropy) alone are insufficient for reliably (i.e., better than random guessing) distinguishing between factual and hallucinated content. To enhance detection, we explore various existing approaches, including prompting-based methods, probing, and fine-tuning, with fine-tuning proving the most effective. To further improve the accuracy, we introduce a new paradigm, named RATE-FT, that augments fine-tuning with an auxiliary task for the model to jointly learn with the main task of hallucination detection. With extensive experiments and analysis using a variety of model families & datasets, we demonstrate the effectiveness and generalizability of our method, e.g., +3% over general fine-tuning methods on LongFact.