UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations

TL;DR

This paper introduces UniConv, a unified model that combines retrieval and response generation for conversational search, enhancing performance by joint fine-tuning and mechanisms to reduce inconsistency.

Contribution

It proposes a novel unified framework for dense retrieval and response generation in conversational systems, addressing limitations of separate models.

Findings

Outperforms existing baselines on five datasets.

Mutually improves retrieval and response generation.

Reduces inconsistency risks in unified modeling.

Abstract

The rapid advancement of conversational search systems revolutionizes how information is accessed by enabling the multi-turn interaction between the user and the system. Existing conversational search systems are usually built with two different models. This separation restricts the system from leveraging the intrinsic knowledge of the models simultaneously, which cannot ensure the effectiveness of retrieval benefiting the generation. The existing studies for developing unified models cannot fully address the aspects of understanding conversational context, managing retrieval independently, and generating responses. In this paper, we explore how to unify dense retrieval and response generation for large language models in conversation. We conduct joint fine-tuning with different objectives and design two mechanisms to reduce the inconsistency risks while mitigating data discrepancy. The evaluations on five conversational search datasets demonstrate that our unified model can mutually improve both tasks and outperform the existing baselines.