Fine-Tuning vs. RAG for Multi-Hop Question Answering with Novel Knowledge

TL;DR

This paper compares fine-tuning and retrieval-augmented generation methods for multi-hop question answering, revealing that retrieval-based approaches excel especially with temporally novel knowledge, while supervised fine-tuning achieves the highest accuracy overall.

Contribution

It systematically evaluates parametric and non-parametric knowledge injection methods on open-source LLMs for multi-hop QA, highlighting the effectiveness of retrieval-augmented generation.

Findings

Retrieval-augmented generation significantly improves accuracy with novel knowledge.

Supervised fine-tuning achieves the highest overall accuracy.

Unsupervised fine-tuning offers limited gains over base models.

Abstract

Multi-hop question answering is widely used to evaluate the reasoning capabilities of large language models (LLMs), as it requires integrating multiple pieces of supporting knowledge to arrive at a correct answer. While prior work has explored different mechanisms for providing knowledge to LLMs, such as finetuning and retrieval-augmented generation (RAG), their relative effectiveness for multi-hop question answering remains insufficiently understood, particularly when the required knowledge is temporally novel. In this paper, we systematically compare parametric and non-parametric knowledge injection methods for open-domain multi-hop question answering. We evaluate unsupervised fine-tuning (continual pretraining), supervised fine-tuning, and retrieval-augmented generation across three 7B-parameter open-source LLMs. Experiments are conducted on two benchmarks: QASC, a standard multi-hop science question answering dataset, and a newly constructed dataset of over 10,000 multi-hop questions derived from Wikipedia events in 2024, designed to test knowledge beyond the models' pretraining cutoff. Our results show that unsupervised fine-tuning provides only limited gains over base models, suggesting that continual pretraining alone is insufficient for improving multi-hop reasoning accuracy. In contrast, retrieval-augmented generation yields substantial and consistent improvements, particularly when answering questions that rely on temporally novel information. Supervised fine-tuning achieves the highest overall accuracy across models and datasets. These findings highlight fundamental differences in how knowledge injection mechanisms support multi-hop question answering and underscore the importance of retrieval-based methods when external or compositional knowledge is required.