Graph-Aware Late Chunking for Retrieval-Augmented Generation in Biomedical Literature

TL;DR

This paper introduces GraLC-RAG, a graph-aware framework for biomedical literature retrieval that improves structural coverage across document sections, addressing limitations of traditional ranking metrics.

Contribution

It proposes a novel structure-aware chunking and retrieval method that enhances multi-section evidence retrieval in biomedical RAG systems.

Findings

Structure-aware methods retrieve from more sections than content-similarity methods.

KG-infused retrieval narrows answer quality gaps while increasing section diversity.

Standard metrics undervalue the importance of structural retrieval in biomedical documents.

Abstract

Retrieval-Augmented Generation (RAG) systems for biomedical literature are typically evaluated using ranking metrics like Mean Reciprocal Rank (MRR), which measure how well the system identifies the single most relevant chunk. We argue that for full-text scientific documents, this paradigm is incomplete: it rewards retrieval precision while ignoring retrieval breadth -- the ability to surface evidence from across a document's structural sections. We propose GraLC-RAG, a framework that unifies late chunking with graph-aware structural intelligence, introducing structure-aware chunk boundary detection, UMLS knowledge graph infusion, and graph-guided hybrid retrieval. We evaluate six strategies on 2,359 IMRaD-filtered PubMed Central articles using 2,033 cross-section questions and two metric families: standard ranking metrics (MRR, Recall@k) and structural coverage metrics (SecCov@k, CS Recall). Our results expose a sharp divergence: content-similarity methods achieve the highest MRR (0.517) but always retrieve from a single section, while structure-aware methods retrieve from up to 15.6x more sections. Generation experiments show that KG-infused retrieval narrows the answer-quality gap to delta-F1 = 0.009 while maintaining 4.6x section diversity. These findings demonstrate that standard metrics systematically undervalue structural retrieval and that closing the multi-section synthesis gap is a key open problem for biomedical RAG.