Answering the Wrong Question: Reasoning Trace Inversion for Abstention in LLMs

TL;DR

This paper introduces Trace Inversion, a novel abstention method for LLMs that detects when models answer the wrong question by comparing initial and reconstructed queries, significantly improving abstention accuracy.

Contribution

The paper proposes the Query Misalignment Framework and Trace Inversion technique, a new approach that enhances LLM abstention by analyzing reasoning traces to identify incorrect answers.

Findings

Trace Inversion outperforms baselines in 33 of 36 settings.

It improves abstention performance across four large language models.

The method effectively detects wrong-question answers using reasoning trace comparison.

Abstract

For Large Language Models (LLMs) to be reliably deployed, models must effectively know when not to answer: abstain. Reasoning models, in particular, have gained attention for impressive performance on complex tasks. However, reasoning models have been shown to have worse abstention abilities. Taking the vulnerabilities of reasoning models into account, we propose our Query Misalignment Framework. Hallucinations resulting in failed abstention can be reinterpreted as LLMs answering the wrong question (rather than answering a question incorrectly). Based on this framework, we develop a new class of state-of-the-art abstention methods called Trace Inversion. First, we generate the reasoning trace of a model. Based on only the trace, we then reconstruct the most likely query that the model responded to. Finally, we compare the initial query with the reconstructed query. Low similarity score between the initial query and reconstructed query suggests that the model likely answered the question incorrectly and is flagged to abstain. Extensive experiments demonstrate that Trace Inversion effectively boosts abstention performance in four frontier LLMs across nine abstention QA datasets, beating competitive baselines in 33 out of 36 settings.