Reranker Optimization via Geodesic Distances on k-NN Manifolds

TL;DR

Maniscope is a geometric reranking method that uses geodesic distances on k-NN manifolds to improve retrieval accuracy and speed in RAG systems, outperforming existing methods with lower latency.

Contribution

It introduces a novel manifold-based reranking approach that captures semantic structure more effectively than flat metrics, enabling faster and more accurate retrieval.

Findings

Outperforms HNSW baseline on challenging datasets with higher NDCG@3.

Achieves comparable accuracy to cross-encoders at significantly lower latency.

Provides a scalable, open-source solution for real-time retrieval-augmented generation.

Abstract

Current neural reranking approaches for retrieval-augmented generation (RAG) rely on cross-encoders or large language models (LLMs), requiring substantial computational resources and exhibiting latencies of 3-5 seconds per query. We propose Maniscope, a geometric reranking method that computes geodesic distances on k-nearest neighbor (k-NN) manifolds constructed over retrieved document candidates. This approach combines global cosine similarity with local manifold geometry to capture semantic structure that flat Euclidean metrics miss. Evaluating on eight BEIR benchmark datasets (1,233 queries), Maniscope outperforms HNSW graph-based baseline on the three hardest datasets (NFCorpus: +7.0%, TREC-COVID: +1.6%, AorB: +2.8% NDCG@3) while being 3.2x faster (4.7 ms vs 14.8 ms average). Compared to cross-encoder rerankers, Maniscope achieves within 2% accuracy at 10-45x lower latency. On TREC-COVID, LLM-Reranker provides only +0.5% NDCG@3 improvement over Maniscope at 840x higher latency, positioning Maniscope as a practical alternative for real-time RAG deployment. The method requires O(N D + M^2 D + M k log k) complexity where M << N , enabling sub-10 ms latency. We plan to release Maniscope as open-source software.