Data Augmentation for Sample Efficient and Robust Document Ranking

TL;DR

This paper introduces data augmentation techniques combined with contrastive learning to improve the sample efficiency and robustness of document ranking models, especially in low-data and out-of-domain scenarios.

Contribution

The paper proposes novel supervised and unsupervised data augmentation methods and adapts contrastive losses for robust, sample-efficient document ranking.

Findings

Data augmentation improves ranking performance across dataset sizes.

Augmented models show enhanced robustness in out-of-domain benchmarks.

Contrastive losses effectively leverage augmented data for better ranking.

Abstract

Contextual ranking models have delivered impressive performance improvements over classical models in the document ranking task. However, these highly over-parameterized models tend to be data-hungry and require large amounts of data even for fine-tuning. In this paper, we propose data-augmentation methods for effective and robust ranking performance. One of the key benefits of using data augmentation is in achieving sample efficiency or learning effectively when we have only a small amount of training data. We propose supervised and unsupervised data augmentation schemes by creating training data using parts of the relevant documents in the query-document pairs. We then adapt a family of contrastive losses for the document ranking task that can exploit the augmented data to learn an effective ranking model. Our extensive experiments on subsets of the MS MARCO and TREC-DL test sets show that data augmentation, along with the ranking-adapted contrastive losses, results in performance improvements under most dataset sizes. Apart from sample efficiency, we conclusively show that data augmentation results in robust models when transferred to out-of-domain benchmarks. Our performance improvements in in-domain and more prominently in out-of-domain benchmarks show that augmentation regularizes the ranking model and improves its robustness and generalization capability.