Dep-Search: Learning Dependency-Aware Reasoning Traces with Persistent Memory

TL;DR

Dep-Search is a novel dependency-aware reasoning framework that improves large language models' multi-hop reasoning by explicitly managing dependencies, retrieval, and persistent memory, leading to significant performance gains.

Contribution

It introduces a structured, dependency-aware search framework with explicit control mechanisms and persistent memory, advancing beyond implicit reasoning approaches in LLMs.

Findings

Significantly improves multi-hop reasoning performance across datasets.

Enhances dependency management and knowledge reuse in LLMs.

Achieves substantial gains over strong baselines.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in complex reasoning tasks, particularly when augmented with search mechanisms that enable systematic exploration of external knowledge bases. The field has evolved from traditional retrieval-augmented generation (RAG) frameworks to more sophisticated search-based frameworks that orchestrate multi-step reasoning through explicit search strategies. However, existing search frameworks still rely heavily on implicit natural language reasoning to determine search strategies and how to leverage retrieved information across reasoning steps. This reliance on implicit reasoning creates fundamental challenges for managing dependencies between sub-questions, efficiently reusing previously retrieved knowledge, and learning optimal search strategies through reinforcement learning. To address these limitations, we propose Dep-Search, a dependency-aware search framework that advances beyond existing search frameworks by integrating structured reasoning, retrieval, and persistent memory through GRPO. Dep-Search introduces explicit control mechanisms that enable the model to decompose questions with dependency relationships, retrieve information when needed, access previously stored knowledge from memory, and summarize long reasoning contexts into reusable memory entries. Through extensive experiments on seven diverse question answering datasets, we demonstrate that Dep-Search significantly enhances LLMs' ability to tackle complex multi-hop reasoning tasks, achieving substantial improvements over strong baselines across different model scales.