RAGShaper: Eliciting Sophisticated Agentic RAG Skills via Automated Data Synthesis

TL;DR

RAGShaper is a data synthesis framework that automates the creation of complex retrieval tasks and agent trajectories, enhancing the robustness of large language models in noisy, real-world environments.

Contribution

It introduces a novel automated data synthesis method with adversarial distractors and constrained navigation to improve agent robustness in retrieval-augmented generation.

Findings

Models trained on RAGShaper data outperform baselines.

Enhanced robustness in noise-intensive retrieval tasks.

Effective in complex problem-solving environments.

Abstract

Agentic Retrieval-Augmented Generation (RAG) empowers large language models to autonomously plan and retrieve information for complex problem-solving. However, the development of robust agents is hindered by the scarcity of high-quality training data that reflects the noise and complexity of real-world retrieval environments. Conventional manual annotation is unscalable and often fails to capture the dynamic reasoning strategies required to handle retrieval failures. To bridge this gap, we introduce RAGShaper, a novel data synthesis framework designed to automate the construction of RAG tasks and robust agent trajectories. RAGShaper incorporates an InfoCurator to build dense information trees enriched with adversarial distractors spanning Perception and Cognition levels. Furthermore, we propose a constrained navigation strategy that forces a teacher agent to confront these distractors, thereby eliciting trajectories that explicitly demonstrate error correction and noise rejection. Comprehensive experiments confirm that models trained on our synthesized corpus significantly outperform existing baselines, exhibiting superior robustness in noise-intensive and complex retrieval tasks.