Hierarchical Retrieval at Scale: Bridging Transparency and Efficiency

TL;DR

This paper introduces Retreever, a scalable hierarchical retrieval method that balances efficiency, accuracy, and interpretability by organizing data in a semantic tree structure optimized for large-scale information retrieval.

Contribution

Retreever is a novel tree-based hierarchical retrieval approach that optimizes structure for performance and offers transparency through semantic groupings, outperforming existing hierarchical methods.

Findings

Retreever achieves high retrieval accuracy at low latency.

It provides meaningful semantic groupings for interpretability.

The method balances cost and utility effectively.

Abstract

Information retrieval is a core component of many intelligent systems as it enables conditioning of outputs on new and large-scale datasets. While effective, the standard practice of encoding data into high-dimensional representations for similarity search entails large memory and compute footprints, and also makes it hard to inspect the inner workings of the system. Hierarchical retrieval methods offer an interpretable alternative by organizing data at multiple granular levels, yet do not match the efficiency and performance of flat retrieval approaches. In this paper, we propose Retreever, a tree-based method that makes hierarchical retrieval viable at scale by directly optimizing its structure for retrieval performance while naturally providing transparency through meaningful semantic groupings. Our method offers the flexibility to balance cost and utility by indexing data using representations from any tree level. We show that Retreever delivers strong coarse (intermediate levels) and fine representations (terminal level), while achieving the highest retrieval accuracy at the lowest latency among hierarchical methods. These results demonstrate that this family of techniques is viable in practical applications.