Reliable Decision Making via Calibration Oriented Retrieval Augmented Generation

TL;DR

This paper introduces CalibRAG, a retrieval method designed to improve the calibration and accuracy of decision-making in Large Language Models by referencing external documents more effectively.

Contribution

The paper proposes CalibRAG, a novel retrieval approach that enhances decision calibration and accuracy in LLMs, addressing limitations of previous RAG methods.

Findings

CalibRAG improves decision calibration across multiple datasets.

CalibRAG enhances the accuracy of generated responses.

Empirical results show superior performance over baseline methods.

Abstract

Recently, Large Language Models (LLMs) have been increasingly used to support various decision-making tasks, assisting humans in making informed decisions. However, when LLMs confidently provide incorrect information, it can lead humans to make suboptimal decisions. To prevent LLMs from generating incorrect information on topics they are unsure of and to improve the accuracy of generated content, prior works have proposed Retrieval Augmented Generation (RAG), where external documents are referenced to generate responses. However, previous RAG methods focus only on retrieving documents most relevant to the input query, without specifically aiming to ensure that the human user's decisions are well-calibrated. To address this limitation, we propose a novel retrieval method called Calibrated Retrieval-Augmented Generation (CalibRAG), which ensures that decisions informed by RAG are well-calibrated. Then we empirically validate that CalibRAG improves calibration performance as well as accuracy, compared to other baselines across various datasets.