MDKeyChunker: Single-Call LLM Enrichment with Rolling Keys and Key-Based Restructuring for High-Accuracy RAG

TL;DR

MDKeyChunker is a pipeline that performs structure-aware chunking, single-call metadata enrichment, and semantic-based restructuring of Markdown documents for high-accuracy retrieval in RAG systems.

Contribution

It introduces a novel three-stage pipeline that enables single-call metadata extraction and structure-aware document restructuring for improved RAG performance.

Findings

Achieves perfect recall@5 with BM25 over structural chunks.

Single-call extraction of seven metadata fields reduces LLM calls.

Dense retrieval pipeline attains high MRR of 0.911 on Markdown corpus.

Abstract

RAG pipelines typically rely on fixed-size chunking, which ignores document structure, fragments semantic units across boundaries, and requires multiple LLM calls per chunk for metadata extraction. We present MDKeyChunker, a three-stage pipeline for Markdown documents that (1) performs structure-aware chunking treating headers, code blocks, tables, and lists as atomic units; (2) enriches each chunk via a single LLM call extracting title, summary, keywords, typed entities, hypothetical questions, and a semantic key, while propagating a rolling key dictionary to maintain document-level context; and (3) restructures chunks by merging those sharing the same semantic key via bin-packing, co-locating related content for retrieval. The single-call design extracts all seven metadata fields in one LLM invocation, eliminating the need for separate per-field extraction passes. Rolling key propagation replaces hand-tuned scoring with LLM-native semantic matching. An empirical evaluation on 30 queries over an 18-document Markdown corpus shows Config D (BM25 over structural chunks) achieves Recall@5=1.000 and MRR=0.911, while dense retrieval over the full pipeline (Config C) reaches Recall@5=0.867. MDKeyChunker is implemented in Python with four dependencies and supports any OpenAI-compatible endpoint.