Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models

TL;DR

The paper presents the ERRR framework, a novel query optimization method for retrieval-augmented large language models that improves accuracy and efficiency by extracting and refining knowledge-specific queries.

Contribution

It introduces a new extract-refine-retrieve-read pipeline with a trainable, distillation-based query optimizer for enhanced RAG system performance.

Findings

ERRR outperforms existing baselines on QA datasets.

The trainable query optimizer reduces computational costs.

ERRR improves retrieval relevance and response accuracy.

Abstract

We introduce the \textit{Extract-Refine-Retrieve-Read} (ERRR) framework, a novel approach designed to bridge the pre-retrieval information gap in Retrieval-Augmented Generation (RAG) systems through query optimization tailored to meet the specific knowledge requirements of Large Language Models (LLMs). Unlike conventional query optimization techniques used in RAG, the ERRR framework begins by extracting parametric knowledge from LLMs, followed by using a specialized query optimizer for refining these queries. This process ensures the retrieval of only the most pertinent information essential for generating accurate responses. Moreover, to enhance flexibility and reduce computational costs, we propose a trainable scheme for our pipeline that utilizes a smaller, tunable model as the query optimizer, which is refined through knowledge distillation from a larger teacher model. Our evaluations on various question-answering (QA) datasets and with different retrieval systems show that ERRR consistently outperforms existing baselines, proving to be a versatile and cost-effective module for improving the utility and accuracy of RAG systems.