CONVERSER: Few-Shot Conversational Dense Retrieval with Synthetic Data Generation

TL;DR

CONVERSER is a framework that leverages large language models to generate synthetic conversational data, enabling effective training of dense retrieval models with minimal in-domain examples, thus reducing data collection costs.

Contribution

The paper introduces a novel few-shot training method for conversational dense retrieval using synthetic data generated by large language models.

Findings

Achieves comparable performance to fully-supervised models on benchmarks

Reduces the need for large in-domain conversational datasets

Demonstrates effectiveness of synthetic data in conversational IR

Abstract

Conversational search provides a natural interface for information retrieval (IR). Recent approaches have demonstrated promising results in applying dense retrieval to conversational IR. However, training dense retrievers requires large amounts of in-domain paired data. This hinders the development of conversational dense retrievers, as abundant in-domain conversations are expensive to collect. In this paper, we propose CONVERSER, a framework for training conversational dense retrievers with at most 6 examples of in-domain dialogues. Specifically, we utilize the in-context learning capability of large language models to generate conversational queries given a passage in the retrieval corpus. Experimental results on conversational retrieval benchmarks OR-QuAC and TREC CAsT 19 show that the proposed CONVERSER achieves comparable performance to fully-supervised models, demonstrating the effectiveness of our proposed framework in few-shot conversational dense retrieval. All source code and generated datasets are available at https://github.com/MiuLab/CONVERSER