Implicit Feedback for Dense Passage Retrieval: A Counterfactual Approach

TL;DR

This paper introduces CoRocchio, a counterfactual method for leveraging implicit feedback in dense passage retrieval, effectively mitigating position bias and improving retrieval accuracy.

Contribution

It proposes the CoRocchio algorithm, a novel counterfactual approach that reduces bias in implicit feedback for dense retrievers, with theoretical and empirical validation.

Findings

CoRocchio produces unbiased dense query representations.

The method improves retrieval effectiveness over baseline models.

Open-source code and experimental framework are provided.

Abstract

In this paper we study how to effectively exploit implicit feedback in Dense Retrievers (DRs). We consider the specific case in which click data from a historic click log is available as implicit feedback. We then exploit such historic implicit interactions to improve the effectiveness of a DR. A key challenge that we study is the effect that biases in the click signal, such as position bias, have on the DRs. To overcome the problems associated with the presence of such bias, we propose the Counterfactual Rocchio (CoRocchio) algorithm for exploiting implicit feedback in Dense Retrievers. We demonstrate both theoretically and empirically that dense query representations learnt with CoRocchio are unbiased with respect to position bias and lead to higher retrieval effectiveness. We make available the implementations of the proposed methods and the experimental framework, along with all results at https://github.com/ielab/Counterfactual-DR.