Assessment of RAG and Fine-Tuning for Industrial Question-Answering-Applications

TL;DR

This paper compares Retrieval-Augmented Generation and fine-tuning for industrial question-answering, finding RAG to be more cost-effective and suitable for open-source models in enterprise contexts.

Contribution

It extends the Cost-of-Pass framework to evaluate RAG and fine-tuning, providing insights into their cost-accuracy trade-offs in industrial QA applications.

Findings

Premium models perform best out of the box.

Open-source models can match quality with RAG.

RAG is the most cost-efficient adaptation method.

Abstract

Large Language Models (LLMs) are increasingly employed in enterprise question-answering (QA) systems, requiring adaptation to domain-specific knowledge. Among the most prevalent methods for incorporating such knowledge are Retrieval-Augmented Generation (RAG) and fine-tuning (FT). Yet, from a cost-accuracy trade-off perspective, it remains unclear which approach best suits industry scenarios. This study examines the impact of RAG and FT on two closed datasets specific to the automotive industry, assessing answer quality and operational costs. We extend the Cost-of-Pass framework proposed by Erol et al. (arXiv:2504.13359) to jointly assess output quality, generation cost, and user interaction cost. Our findings reveal that while premium models perform best out of the box, open-source models can achieve comparable quality when enhanced with RAG. Overall, RAG emerges as the most effective and cost-efficient adaptation method for both closed- and open-source models.