Pause or Fabricate? Training Language Models for Grounded Reasoning

TL;DR

This paper introduces GRIL, a reinforcement learning framework that improves grounded reasoning in language models by enabling them to recognize information gaps and pause for clarification, reducing hallucinations.

Contribution

The paper proposes a novel multi-turn RL approach with stage-specific rewards to enhance premise detection and prevent ungrounded reasoning in language models.

Findings

GRIL improves premise detection accuracy by up to 45%.

Task success increases by 30% with GRIL.

Average response length decreases by over 20% with the proposed method.

Abstract

Large language models have achieved remarkable progress on complex reasoning tasks. However, they often implicitly fabricate information when inputs are incomplete, producing confident but unreliable conclusions -- a failure mode we term ungrounded reasoning. We argue that this issue arises not from insufficient reasoning capability, but from the lack of inferential boundary awareness -- the ability to recognize when the necessary premises for valid inference are missing. To address this issue, we propose Grounded Reasoning via Interactive Reinforcement Learning (GRIL), a multi-turn reinforcement learning framework for grounded reasoning under incomplete information. GRIL decomposes the reasoning process into two stages: clarify and pause, which identifies whether the available information is sufficient, and grounded reasoning, which performs task solving once the necessary premises are established. We design stage-specific rewards to penalize hallucinations, enabling models to detect gaps, stop proactively, and resume reasoning after clarification. Experiments on GSM8K-Insufficient and MetaMATH-Insufficient show that GRIL significantly improves premise detection (up to 45%), leading to a 30% increase in task success while reducing average response length by over 20%. Additional analyses confirm robustness to noisy user responses and generalization to out-of-distribution tasks.