SIGHT: Reinforcement Learning with Self-Evidence and Information-Gain Diverse Branching for Search Agent

TL;DR

SIGHT enhances reinforcement learning for search agents by using self-evidence and information gain to improve reasoning accuracy and reduce redundancy in complex question-answering tasks.

Contribution

It introduces a novel framework combining self-evidence support and information-gain driven diverse branching for more effective search-based reasoning.

Findings

Outperforms existing methods on QA benchmarks.

Reduces search steps needed for accurate answers.

Improves reasoning in multi-hop scenarios.

Abstract

Reinforcement Learning (RL) has empowered Large Language Models (LLMs) to master autonomous search for complex question answering. However, particularly within multi-turn search scenarios, this interaction introduces a critical challenge: search results often suffer from high redundancy and low signal-to-noise ratios. Consequently, agents easily fall into "Tunnel Vision," where the forced interpretation of early noisy retrievals leads to irreversible error accumulation. To address these challenges, we propose SIGHT, a framework that enhances search-based reasoning through Self-Evidence Support (SES) and Information-Gain Driven Diverse Branching. SIGHT distills search results into high-fidelity evidence via SES and calculates an Information Gain score to pinpoint pivotal states where observations maximally reduce uncertainty. This score guides Dynamic Prompting Interventions - including de-duplication, reflection, or adaptive branching - to spawn new branches with SES. Finally, by integrating SES and correctness rewards via Group Relative Policy Optimization, SIGHT internalizes robust exploration strategies without external verifiers. Experiments on single-hop and multi-hop QA benchmarks demonstrate that SIGHT significantly outperforms existing approaches, particularly in complex reasoning scenarios, using fewer search steps.