Chunk Twice, Embed Once: A Systematic Study of Segmentation and Representation Trade-offs in Chemistry-Aware Retrieval-Augmented Generation

TL;DR

This paper systematically evaluates document segmentation and embedding strategies for chemistry-focused Retrieval-Augmented Generation systems, revealing optimal configurations and providing benchmarks to guide future development.

Contribution

It offers the first large-scale analysis of segmentation and embedding choices in chemistry RAG, introducing new datasets and evaluation tools for domain-specific retrieval.

Findings

Recursive token-based chunking (R100-0) performs best.

Retrieval-optimized embeddings outperform domain-specific models.

New benchmark datasets and evaluation framework released.

Abstract

Retrieval-Augmented Generation (RAG) systems are increasingly vital for navigating the ever-expanding body of scientific literature, particularly in high-stakes domains such as chemistry. Despite the promise of RAG, foundational design choices -- such as how documents are segmented and represented -- remain underexplored in domain-specific contexts. This study presents the first large-scale, systematic evaluation of chunking strategies and embedding models tailored to chemistry-focused RAG systems. We investigate 25 chunking configurations across five method families and evaluate 48 embedding models on three chemistry-specific benchmarks, including the newly introduced QuestChemRetrieval dataset. Our results reveal that recursive token-based chunking (specifically R100-0) consistently outperforms other approaches, offering strong performance with minimal resource overhead. We also find that retrieval-optimized embeddings -- such as Nomic and Intfloat E5 variants -- substantially outperform domain-specialized models like SciBERT. By releasing our datasets, evaluation framework, and empirical benchmarks, we provide actionable guidelines for building effective and efficient chemistry-aware RAG systems.