Generalizing Conversational Dense Retrieval via LLM-Cognition Data Augmentation

TL;DR

This paper introduces ConvAug, a novel framework that enhances conversational dense retrieval models by generating diverse, cognition-inspired augmented data and employing contrastive learning, significantly improving generalization across various conversational scenarios.

Contribution

ConvAug is the first to incorporate multi-level data augmentation and cognition-aware filtering for conversational retrieval, addressing data sparsity and improving model robustness.

Findings

Improves retrieval accuracy on four public datasets.

Enhances zero-shot generalization capabilities.

Demonstrates effectiveness of cognition-inspired augmentation.

Abstract

Conversational search utilizes muli-turn natural language contexts to retrieve relevant passages. Existing conversational dense retrieval models mostly view a conversation as a fixed sequence of questions and responses, overlooking the severe data sparsity problem -- that is, users can perform a conversation in various ways, and these alternate conversations are unrecorded. Consequently, they often struggle to generalize to diverse conversations in real-world scenarios. In this work, we propose a framework for generalizing Conversational dense retrieval via LLM-cognition data Augmentation (ConvAug). ConvAug first generates multi-level augmented conversations to capture the diverse nature of conversational contexts. Inspired by human cognition, we devise a cognition-aware process to mitigate the generation of false positives, false negatives, and hallucinations. Moreover, we develop a difficulty-adaptive sample filter that selects challenging samples for complex conversations, thereby giving the model a larger learning space. A contrastive learning objective is then employed to train a better conversational context encoder. Extensive experiments conducted on four public datasets, under both normal and zero-shot settings, demonstrate the effectiveness, generalizability, and applicability of ConvAug. The code is released at https://github.com/haon-chen/ConvAug.