Hypencoder: Hypernetworks for Information Retrieval
Julian Killingback, Hansi Zeng, Hamed Zamani

TL;DR
Hypencoder introduces a novel retrieval paradigm using hypernetworks to generate query-specific relevance functions, significantly improving retrieval performance and efficiency over traditional vector-based methods.
Contribution
This paper presents Hypencoders, a new class of models that use hypernetworks to produce query-specific relevance functions, surpassing existing dense retrieval and reranking models.
Findings
Hypencoders outperform strong dense retrieval models.
Hypencoders surpass reranking models with fewer parameters.
Efficient retrieval from 8.8M documents in under 60ms.
Abstract
Existing information retrieval systems are largely constrained by their reliance on vector inner products to assess query-document relevance, which naturally limits the expressiveness of the relevance score they can produce. We propose a new paradigm; instead of representing a query as a vector, we use a small neural network that acts as a learned query-specific relevance function. This small neural network takes a document representation as input (in this work we use a single vector) and produces a scalar relevance score. To produce the small neural network we use a hypernetwork, a network that produces the weights of other networks, as our query encoder. We name this category of encoder models Hypencoders. Experiments on in-domain search tasks show that Hypencoders significantly outperform strong dense retrieval models and even surpass reranking models and retrieval models with an…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsComputational Physics and Python Applications · Neural Networks and Applications · Time Series Analysis and Forecasting
MethodsSparse Evolutionary Training
