RAGtifier: Evaluating RAG Generation Approaches of State-of-the-Art RAG Systems for the SIGIR LiveRAG Competition

TL;DR

This paper evaluates various Retrieval-Augmented Generation (RAG) approaches in the context of the SIGIR LiveRAG 2025 challenge, demonstrating a competitive solution that combines specific retrievers and rerankers to enhance factual correctness and faithfulness.

Contribution

It presents an empirical analysis of RAG systems under challenge conditions and introduces a top-performing solution using InstructRAG, Pinecone, and BGE reranker.

Findings

Achieved third place in the SIGIR 2025 LiveRAG Challenge.

Demonstrated effectiveness of combining InstructRAG with Pinecone retriever.

Provided insights into retriever and reranker combinations for RAG systems.

Abstract

Retrieval-Augmented Generation (RAG) enriches Large Language Models (LLMs) by combining their internal, parametric knowledge with external, non-parametric sources, with the goal of improving factual correctness and minimizing hallucinations. The LiveRAG 2025 challenge explores RAG solutions to maximize accuracy on DataMorgana's QA pairs, which are composed of single-hop and multi-hop questions. The challenge provides access to sparse OpenSearch and dense Pinecone indices of the Fineweb 10BT dataset. It restricts model use to LLMs with up to 10B parameters and final answer generation with Falcon-3-10B. A judge-LLM assesses the submitted answers along with human evaluators. By exploring distinct retriever combinations and RAG solutions under the challenge conditions, our final solution emerged using InstructRAG in combination with a Pinecone retriever and a BGE reranker. Our solution achieved a correctness score of 1.13 and a faithfulness score of 0.55 in the non-human evaluation, placing it overall in third place in the SIGIR 2025 LiveRAG Challenge.