RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization

TL;DR

RoseRAG introduces a novel framework that improves small-scale LLMs' robustness in retrieval-augmented generation by using margin-aware preference optimization, multi-turn prompting, and rejection sampling, leading to superior performance in open-domain QA tasks.

Contribution

The paper presents RoseRAG, a new robust RAG framework for small-scale LLMs that incorporates margin-aware optimization and preference-based refinement to enhance accuracy and reliability.

Findings

Outperforms state-of-the-art baselines on three QA benchmarks.

Enhances response quality through contrastive preference selection.

Improves robustness of small-scale LLMs in retrieval-augmented tasks.

Abstract

Large language models (LLMs) have achieved impressive performance but face high computational costs and latency, limiting their deployment in resource-constrained settings. In contrast, small-scale LLMs (SLMs) are more efficient yet struggle to capture evolving real-world knowledge. Retrieval-augmented generation (RAG) helps by integrating external knowledge, but imperfect retrieval can introduce distracting noise that misleads SLMs. We propose RoseRAG, a robust RAG framework for SLMs via Margin-aware Preference Optimization. RoseRAG employs multi-turn prompting for detailed reasoning, rejection sampling for high-quality explanations, and contrastive preference selection to refine responses by maximizing the likelihood gap between preferred and non-preferred outputs. By integrating these components into a margin-aware optimization process, RoseRAG robustly enhances the accuracy and reliability of SLMs for RAG applications. Extensive experiments on three open-domain question answering benchmarks indicate that our innovative RoseRAG surpasses state-of-the-art baselines significantly.