Harnessing Reasoning Trajectories for Hallucination Detection via Answer-agreement Representation Shaping

TL;DR

This paper introduces ARS, a method that improves hallucination detection in large reasoning models by shaping trace representations through answer agreement and counterfactual perturbations.

Contribution

ARS is a novel approach that learns detection-friendly representations by explicitly encoding answer stability without requiring human annotations.

Findings

ARS improves hallucination detection accuracy.

ARS outperforms strong baseline detectors.

ARS requires no human annotations during training.

Abstract

Large reasoning models (LRMs) often generate long, seemingly coherent reasoning traces yet still produce incorrect answers, making hallucination detection challenging. Although trajectories contain useful signals, directly using trace text or vanilla hidden states for detection is brittle: traces vary in form and detectors can overfit to superficial patterns rather than answer validity. We introduce Answer-agreement Representation Shaping (ARS), which learns detection-friendly trace-conditioned representations by explicitly encoding answer stability. ARS generates counterfactual answers through small latent interventions, specifically, perturbing the trace-boundary embedding, and labels each perturbation by whether the resulting answer agrees with the original. It then learns representations that bring answer-agreeing states together and separate answer-disagreeing ones, exposing latent instability indicative of hallucination risk. The shaped embeddings are plug-and-play with existing embedding-based detectors and require no human annotations during training. Experiments demonstrate that ARS consistently improves detection and achieves substantial gains over strong baselines. Code is available at: https://github.com/radiolab-ntu/ars_icml2026.