Augmenting Document Representations for Dense Retrieval with Interpolation and Perturbation

TL;DR

This paper introduces DAR, a document augmentation framework using interpolation and perturbation to improve dense retrieval performance, especially when labeled data is scarce.

Contribution

The paper proposes a novel document augmentation method for dense retrieval that enhances representations through interpolation and perturbation, improving performance on benchmark datasets.

Findings

DAR significantly outperforms baseline models on dense retrieval tasks.

The augmentation method benefits both labeled and unlabeled document retrieval.

Experimental results demonstrate improved retrieval accuracy with DAR.

Abstract

Dense retrieval models, which aim at retrieving the most relevant document for an input query on a dense representation space, have gained considerable attention for their remarkable success. Yet, dense models require a vast amount of labeled training data for notable performance, whereas it is often challenging to acquire query-document pairs annotated by humans. To tackle this problem, we propose a simple but effective Document Augmentation for dense Retrieval (DAR) framework, which augments the representations of documents with their interpolation and perturbation. We validate the performance of DAR on retrieval tasks with two benchmark datasets, showing that the proposed DAR significantly outperforms relevant baselines on the dense retrieval of both the labeled and unlabeled documents.