The Geometry of Queries: Query-Based Innovations in Retrieval-Augmented Generation for Healthcare QA

TL;DR

This paper presents QB-RAG, a novel retrieval-augmented generation framework for healthcare QA that improves accuracy by pre-aligning user queries with curated, answerable questions using an LLM-based filtering mechanism.

Contribution

It introduces QB-RAG, a new method that enhances healthcare question answering by ensuring relevant, answerable queries are used, backed by a comprehensive evaluation framework.

Findings

QB-RAG outperforms existing retrieval methods in healthcare QA.

The filtering mechanism improves relevance and answerability of retrieved questions.

Empirical results show increased accuracy and trustworthiness in healthcare applications.

Abstract

Deploying Large Language Models (LLMs) for healthcare question answering requires robust methods to ensure accuracy and reliability. This work introduces Query-Based Retrieval Augmented Generation (QB-RAG), a framework for enhancing Retrieval-Augmented Generation (RAG) systems in healthcare question-answering by pre-aligning user queries with a database of curated, answerable questions derived from healthcare content. A key component of QB-RAG is an LLM-based filtering mechanism that ensures that only relevant and answerable questions are included in the database, enabling reliable reference query generation at scale. We provide theoretical motivation for QB-RAG, conduct a comparative analysis of existing retrieval enhancement techniques, and introduce a generalizable, comprehensive evaluation framework that assesses both the retrieval effectiveness and the quality of the generated response based on faithfulness, relevance, and adherence to the guideline. Our empirical evaluation on a healthcare data set demonstrates the superior performance of QB-RAG compared to existing retrieval methods, highlighting its practical value in building trustworthy digital health applications for health question-answering.