RMIT-ADM+S at the SIGIR 2025 LiveRAG Challenge

TL;DR

This paper describes the winning system for the SIGIR 2025 LiveRAG Challenge, which uses a Generation-Retrieval-Augmented Generation approach with systematic evaluation to improve answer quality.

Contribution

The paper introduces a novel G-RAG approach with a pointwise LLM re-ranking step and a comprehensive evaluation framework for retrieval-augmented generation systems.

Findings

Achieved the highest Borda score in the challenge.

Systematic evaluation identified optimal configurations.

Demonstrated effectiveness of G-RAG with re-ranking.

Abstract

This paper presents the RMIT--ADM+S winning system in the SIGIR 2025 LiveRAG Challenge. Our Generation-Retrieval-Augmented Generation (G-RAG) approach generates a hypothetical answer that is used during the retrieval phase, alongside the original question. G-RAG also incorporates a pointwise large language model (LLM)-based re-ranking step prior to final answer generation. We describe the system architecture and the rationale behind our design choices. In particular, a systematic evaluation using the Grid of Points approach and N-way ANOVA enabled a controlled comparison of multiple configurations, including query variant generation, question decomposition, rank fusion strategies, and prompting techniques for answer generation. The submitted system achieved the highest Borda score based on the aggregation of Coverage, Relatedness, and Quality scores from manual evaluations, ranking first in the SIGIR 2025 LiveRAG Challenge.