Out-of-Domain Semantics to the Rescue! Zero-Shot Hybrid Retrieval Models
Tao Chen, Mingyang Zhang, Jing Lu, Michael Bendersky, Marc Najork

TL;DR
This paper investigates the generalization challenges of deep retrieval models in out-of-domain scenarios and proposes a hybrid approach combining lexical and deep models, resulting in significant performance improvements.
Contribution
It introduces a simple hybrid framework that effectively combines lexical and deep retrieval models to enhance out-of-domain passage retrieval performance.
Findings
Deep models' performance drops significantly out-of-domain.
Lexical models are more robust across different domains.
Hybrid models outperform individual models in out-of-domain settings.
Abstract
The pre-trained language model (eg, BERT) based deep retrieval models achieved superior performance over lexical retrieval models (eg, BM25) in many passage retrieval tasks. However, limited work has been done to generalize a deep retrieval model to other tasks and domains. In this work, we carefully select five datasets, including two in-domain datasets and three out-of-domain datasets with different levels of domain shift, and study the generalization of a deep model in a zero-shot setting. Our findings show that the performance of a deep retrieval model is significantly deteriorated when the target domain is very different from the source domain that the model was trained on. On the contrary, lexical models are more robust across domains. We thus propose a simple yet effective framework to integrate lexical and deep retrieval models. Our experiments demonstrate that these two models…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Multimodal Machine Learning Applications · Domain Adaptation and Few-Shot Learning
