Evaluating Self-Generated Documents for Enhancing Retrieval-Augmented Generation with Large Language Models

TL;DR

This paper investigates the effectiveness and properties of self-generated documents in retrieval-augmented generation systems, providing a taxonomy and strategies to improve knowledge-intensive question answering.

Contribution

It offers a comprehensive analysis of Self-Docs, introduces a taxonomy based on Systemic Functional Linguistics, and proposes strategies for their effective integration with external sources.

Findings

Certain Self-Doc types significantly improve RAG performance

A taxonomy categorizes Self-Docs based on linguistic properties

Guidelines for combining Self-Docs with external sources are provided

Abstract

The integration of documents generated by LLMs themselves (Self-Docs) alongside retrieved documents has emerged as a promising strategy for retrieval-augmented generation systems. However, previous research primarily focuses on optimizing the use of Self-Docs, with their inherent properties remaining underexplored. To bridge this gap, we first investigate the overall effectiveness of Self-Docs, identifying key factors that shape their contribution to RAG performance (RQ1). Building on these insights, we develop a taxonomy grounded in Systemic Functional Linguistics to compare the influence of various Self-Docs categories (RQ2) and explore strategies for combining them with external sources (RQ3). Our findings reveal which types of Self-Docs are most beneficial and offer practical guidelines for leveraging them to achieve significant improvements in knowledge-intensive question answering tasks.