Breaking It Down: Domain-Aware Semantic Segmentation for Retrieval Augmented Generation

TL;DR

This paper presents two novel semantic chunking methods, PSC and MFC, that significantly improve retrieval and generation quality in domain-specific document processing, with strong out-of-domain generalization.

Contribution

Introduction of two efficient semantic chunking methods, PSC and MFC, trained on PubMed data, with an evaluation framework demonstrating their impact on retrieval and generation.

Findings

Substantial retrieval improvements (24x with PSC) in MRR and Hits@k.

PSC and MFC generalize well across multiple datasets.

PSC consistently delivers superior performance.

Abstract

Document chunking is a crucial component of Retrieval-Augmented Generation (RAG), as it directly affects the retrieval of relevant and precise context. Conventional fixed-length and recursive splitters often produce arbitrary, incoherent segments that fail to preserve semantic structure. Although semantic chunking has gained traction, its influence on generation quality remains underexplored. This paper introduces two efficient semantic chunking methods, Projected Similarity Chunking (PSC) and Metric Fusion Chunking (MFC), trained on PubMed data using three different embedding models. We further present an evaluation framework that measures the effect of chunking on both retrieval and generation by augmenting PubMedQA with full-text PubMed Central articles. Our results show substantial retrieval improvements (24x with PSC) in MRR and higher Hits@k on PubMedQA. We provide a comprehensive analysis, including statistical significance and response-time comparisons with common chunking libraries. Despite being trained on a single domain, PSC and MFC also generalize well, achieving strong out-of-domain generation performance across multiple datasets. Overall, our findings confirm that our semantic chunkers, especially PSC, consistently deliver superior performance.