Less Context, Same Performance: A RAG Framework for Resource-Efficient LLM-Based Clinical NLP

TL;DR

This paper demonstrates that a Retrieval Augmented Generation (RAG) approach using relevant text segments can match the performance of full clinical notes processing in LLM-based clinical NLP, reducing computational costs.

Contribution

It introduces a RAG framework that efficiently handles long clinical texts by retrieving relevant segments, maintaining accuracy while reducing token usage.

Findings

RAG achieves similar accuracy to full text processing

Significant reduction in token usage with RAG

No statistically significant difference in performance metrics

Abstract

Long text classification is challenging for Large Language Models (LLMs) due to token limits and high computational costs. This study explores whether a Retrieval Augmented Generation (RAG) approach using only the most relevant text segments can match the performance of processing entire clinical notes with large context LLMs. We begin by splitting clinical documents into smaller chunks, converting them into vector embeddings, and storing these in a FAISS index. We then retrieve the top 4,000 words most pertinent to the classification query and feed these consolidated segments into an LLM. We evaluated three LLMs (GPT4o, LLaMA, and Mistral) on a surgical complication identification task. Metrics such as AUC ROC, precision, recall, and F1 showed no statistically significant differences between the RAG based approach and whole-text processing (p > 0.05p > 0.05). These findings indicate that RAG can significantly reduce token usage without sacrificing classification accuracy, providing a scalable and cost effective solution for analyzing lengthy clinical documents.