Lightweight Relevance Grader in RAG

TL;DR

This paper introduces a lightweight relevance grader based on fine-tuned Llama-3.2-1b for RAG systems, significantly improving relevance verification efficiency and precision while maintaining competitive performance.

Contribution

The work fine-tunes a small language model as a relevance grader, achieving high precision comparable to larger models, reducing computational costs in RAG systems.

Findings

Precision increased from 0.1301 to 0.7750

Comparable to llama-3.1-70b in relevance accuracy

Code is publicly available for reproducibility

Abstract

Retrieval-Augmented Generation (RAG) addresses limitations of large language models (LLMs) by leveraging a vector database to provide more accurate and up-to-date information. When a user submits a query, RAG executes a vector search to find relevant documents, which are then used to generate a response. However, ensuring the relevance of retrieved documents with a query would be a big challenge. To address this, a secondary model, known as a relevant grader, can be served to verify its relevance. To reduce computational requirements of a relevant grader, a lightweight small language model is preferred. In this work, we finetuned llama-3.2-1b as a relevant grader and achieved a significant increase in precision from 0.1301 to 0.7750. Its precision is comparable to that of llama-3.1-70b. Our code is available at https://github.com/taeheej/Lightweight-Relevance-Grader-in-RAG.