ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation

TL;DR

ParetoRAG introduces a sentence-level refinement framework for RAG systems that improves retrieval accuracy and generation quality by dynamically re-weighting core content, validated across multiple datasets and models.

Contribution

It proposes an unsupervised, sentence-level optimization method for RAG systems that enhances efficiency and relevance without extra training or API costs.

Findings

Improves retrieval precision and generation quality

Validated across various datasets and models

Does not require additional training or API resources

Abstract

While Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by incorporating external knowledge, they still face persistent challenges in retrieval inefficiency and the inability of LLMs to filter out irrelevant information. We present ParetoRAG, an unsupervised framework that optimizes RAG systems through sentence-level refinement guided by the Pareto principle. By decomposing paragraphs into sentences and dynamically re-weighting core content while preserving contextual coherence, ParetoRAG achieves dual improvements in both retrieval precision and generation quality without requiring additional training or API resources. This framework has been empirically validated across various datasets, LLMs, and retrievers.