Unsupervised dense retrieval with conterfactual contrastive learning

TL;DR

This paper introduces a novel unsupervised counterfactual contrastive learning approach to enhance the robustness and explainability of dense retrieval models in information retrieval tasks, effectively identifying relevant passages without labeled data.

Contribution

It proposes a new counterfactual regularization method based on game theory and unsupervised learning, improving robustness and interpretability of dense retrieval models.

Findings

Enhanced robustness against adversarial attacks.

Ability to identify key relevant passages without supervision.

Outperforms existing anti-attack methods.

Abstract

Efficiently retrieving a concise set of candidates from a large document corpus remains a pivotal challenge in Information Retrieval (IR). Neural retrieval models, particularly dense retrieval models built with transformers and pretrained language models, have been popular due to their superior performance. However, criticisms have also been raised on their lack of explainability and vulnerability to adversarial attacks. In response to these challenges, we propose to improve the robustness of dense retrieval models by enhancing their sensitivity of fine-graned relevance signals. A model achieving sensitivity in this context should exhibit high variances when documents' key passages determining their relevance to queries have been modified, while maintaining low variances for other changes in irrelevant passages. This sensitivity allows a dense retrieval model to produce robust results with respect to attacks that try to promote documents without actually increasing their relevance. It also makes it possible to analyze which part of a document is actually relevant to a query, and thus improve the explainability of the retrieval model. Motivated by causality and counterfactual analysis, we propose a series of counterfactual regularization methods based on game theory and unsupervised learning with counterfactual passages. Experiments show that, our method can extract key passages without reliance on the passage-level relevance annotations. Moreover, the regularized dense retrieval models exhibit heightened robustness against adversarial attacks, surpassing the state-of-the-art anti-attack methods.