Data-Efficient Autoregressive Document Retrieval for Fact Verification

TL;DR

This paper presents a data-efficient autoregressive document retrieval method for fact verification that does not require annotated data and achieves competitive results, with potential to surpass fully supervised models using less data.

Contribution

Introduces a distant-supervision training approach for autoregressive retrievers that performs well without annotations and approaches supervised performance with minimal labeled data.

Findings

Achieves competitive R-Precision and Recall in zero-shot settings.

Fine-tuning with limited data can match or surpass fully supervised models.

Reduces annotation requirements for effective document retrieval.

Abstract

Document retrieval is a core component of many knowledge-intensive natural language processing task formulations such as fact verification and question answering. Sources of textual knowledge, such as Wikipedia articles, condition the generation of answers from the models. Recent advances in retrieval use sequence-to-sequence models to incrementally predict the title of the appropriate Wikipedia page given a query. However, this method requires supervision in the form of human annotation to label which Wikipedia pages contain appropriate context. This paper introduces a distant-supervision method that does not require any annotation to train autoregressive retrievers that attain competitive R-Precision and Recall in a zero-shot setting. Furthermore we show that with task-specific supervised fine-tuning, autoregressive retrieval performance for two Wikipedia-based fact verification tasks can approach or even exceed full supervision using less than $1/4$ of the annotated data indicating possible directions for data-efficient autoregressive retrieval.