Beyond Nearest Neighbors: Semantic Compression and Graph-Augmented Retrieval for Enhanced Vector Search

TL;DR

This paper introduces semantic compression and graph-augmented retrieval to improve vector search by enhancing diversity, coverage, and contextual understanding beyond traditional nearest neighbor methods.

Contribution

It formalizes semantic compression using submodular optimization and information geometry, and proposes graph-augmented retrieval for multi-hop, context-aware search.

Findings

Semantic compression captures broader semantic structures.

Graph-augmented retrieval enables multi-hop, context-aware search.

Theoretical analysis shows limitations of proximity-based retrieval.

Abstract

Vector databases typically rely on approximate nearest neighbor (ANN) search to retrieve the top-k closest vectors to a query in embedding space. While effective, this approach often yields semantically redundant results, missing the diversity and contextual richness required by applications such as retrieval-augmented generation (RAG), multi-hop QA, and memory-augmented agents. We introduce a new retrieval paradigm: semantic compression, which aims to select a compact, representative set of vectors that captures the broader semantic structure around a query. We formalize this objective using principles from submodular optimization and information geometry, and show that it generalizes traditional top-k retrieval by prioritizing coverage and diversity. To operationalize this idea, we propose graph-augmented vector retrieval, which overlays semantic graphs (e.g., kNN or knowledge-based links) atop vector spaces to enable multi-hop, context-aware search. We theoretically analyze the limitations of proximity-based retrieval under high-dimensional concentration and highlight how graph structures can improve semantic coverage. Our work outlines a foundation for meaning-centric vector search systems, emphasizing hybrid indexing, diversity-aware querying, and structured semantic retrieval. We make our implementation publicly available to foster future research in this area.