SSP: Self-Supervised Post-training for Conversational Search

TL;DR

This paper introduces SSP, a self-supervised post-training method that enhances conversational search models by improving dialogue structure understanding and semantic comprehension, leading to better performance on benchmark datasets.

Contribution

The paper proposes a novel self-supervised post-training paradigm that can be integrated into existing models to improve conversational search performance.

Findings

Boosts performance on CAsT-19 and CAsT-20 datasets.

Enhances understanding of dialogue structure and semantics.

Applicable to most existing conversational models.

Abstract

Conversational search has been regarded as the next-generation search paradigm. Constrained by data scarcity, most existing methods distill the well-trained ad-hoc retriever to the conversational retriever. However, these methods, which usually initialize parameters by query reformulation to discover contextualized dependency, have trouble in understanding the dialogue structure information and struggle with contextual semantic vanishing. In this paper, we propose \fullmodel (\model) which is a new post-training paradigm with three self-supervised tasks to efficiently initialize the conversational search model to enhance the dialogue structure and contextual semantic understanding. Furthermore, the \model can be plugged into most of the existing conversational models to boost their performance. To verify the effectiveness of our proposed method, we apply the conversational encoder post-trained by \model on the conversational search task using two benchmark datasets: CAsT-19 and CAsT-20. Extensive experiments that our \model can boost the performance of several existing conversational search methods. Our source code is available at \url{https://github.com/morecry/SSP}.