Dynamic and Parametric Retrieval-Augmented Generation

TL;DR

This paper reviews recent advances in Dynamic and Parametric Retrieval-Augmented Generation, highlighting their roles in improving knowledge access and integration in large language models for complex tasks.

Contribution

It provides a comprehensive overview of the emerging research areas of Dynamic and Parametric RAG, including theoretical foundations and practical insights.

Findings

Dynamic RAG enables real-time, adaptive retrieval during generation.

Parametric RAG shifts knowledge injection from input to parameter level.

The tutorial summarizes recent advances and practical insights in RAG.

Abstract

Retrieval-Augmented Generation (RAG) has become a foundational paradigm for equipping large language models (LLMs) with external knowledge, playing a critical role in information retrieval and knowledge-intensive applications. However, conventional RAG systems typically adopt a static retrieve-then-generate pipeline and rely on in-context knowledge injection, which can be suboptimal for complex tasks that require multihop reasoning, adaptive information access, and deeper integration of external knowledge. Motivated by these limitations, the research community has moved beyond static retrieval and in-context knowledge injection. Among the emerging directions, this tutorial delves into two rapidly growing and complementary research areas on RAG: Dynamic RAG and Parametric RAG. Dynamic RAG adaptively determines when and what to retrieve during the LLM's generation process, enabling real-time adaptation to the LLM's evolving information needs. Parametric RAG rethinks how retrieved knowledge should be injected into LLMs, transitioning from input-level to parameter-level knowledge injection for enhanced efficiency and effectiveness. This tutorial offers a comprehensive overview of recent advances in these emerging research areas. It also shares theoretical foundations and practical insights to support and inspire further research in RAG.