Multi-task Retrieval for Knowledge-Intensive Tasks

TL;DR

This paper introduces a multi-task neural retrieval model that is robust across various knowledge-intensive tasks, outperforming previous methods especially in few-shot scenarios and matching state-of-the-art results.

Contribution

The paper presents a universal multi-task trained neural retrieval model that improves robustness and performance across diverse tasks and data regimes.

Findings

Outperforms previous retrieval methods in few-shot settings

Rivals specialized neural retrievers with abundant in-domain data

Enhances downstream task models and achieves state-of-the-art results

Abstract

Retrieving relevant contexts from a large corpus is a crucial step for tasks such as open-domain question answering and fact checking. Although neural retrieval outperforms traditional methods like tf-idf and BM25, its performance degrades considerably when applied to out-of-domain data. Driven by the question of whether a neural retrieval model can be universal and perform robustly on a wide variety of problems, we propose a multi-task trained model. Our approach not only outperforms previous methods in the few-shot setting, but also rivals specialised neural retrievers, even when in-domain training data is abundant. With the help of our retriever, we improve existing models for downstream tasks and closely match or improve the state of the art on multiple benchmarks.