Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision
Si Sun, Yingzhuo Qian, Zhenghao Liu, Chenyan Xiong, Kaitao Zhang, Jie, Bao, Zhiyuan Liu, Paul Bennett

TL;DR
MetaAdaptRank is a domain adaptive learning method that enhances neural information retrieval in few-shot scenarios by synthesizing and reweighting weak supervision signals from source domains, significantly improving ranking accuracy.
Contribution
The paper introduces MetaAdaptRank, a novel meta-learning approach that synthesizes and reweights weak supervision signals for effective domain adaptation in few-shot neural IR tasks.
Findings
Significant improvement in few-shot ranking accuracy across three TREC benchmarks.
MetaAdaptRank benefits from contrastive weak data synthesis.
MetaAdaptRank's effectiveness is due to its meta-reweighted data selection.
Abstract
The effectiveness of Neural Information Retrieval (Neu-IR) often depends on a large scale of in-domain relevance training signals, which are not always available in real-world ranking scenarios. To democratize the benefits of Neu-IR, this paper presents MetaAdaptRank, a domain adaptive learning method that generalizes Neu-IR models from label-rich source domains to few-shot target domains. Drawing on source-domain massive relevance supervision, MetaAdaptRank contrastively synthesizes a large number of weak supervision signals for target domains and meta-learns to reweight these synthetic "weak" data based on their benefits to the target-domain ranking accuracy of Neu-IR models. Experiments on three TREC benchmarks in the web, news, and biomedical domains show that MetaAdaptRank significantly improves the few-shot ranking accuracy of Neu-IR models. Further analyses indicate that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Text and Document Classification Technologies · Information Retrieval and Search Behavior
