Zero-Shot Dialogue Disentanglement by Self-Supervised Entangled Response Selection

TL;DR

This paper introduces a zero-shot method for dialogue disentanglement that leverages a response selection model trained on unlabeled web data, achieving promising results without requiring annotated reply-to links.

Contribution

It presents the first zero-shot dialogue disentanglement approach using a self-supervised response selection model trained on unlabeled data, reducing annotation effort.

Findings

Model achieves a cluster F1 score of 25 without labeled data.

Fine-tuning with 10% labeled data nearly matches full data performance.

Code is publicly available for reproducibility.

Abstract

Dialogue disentanglement aims to group utterances in a long and multi-participant dialogue into threads. This is useful for discourse analysis and downstream applications such as dialogue response selection, where it can be the first step to construct a clean context/response set. Unfortunately, labeling all~\emph{reply-to} links takes quadratic effort w.r.t the number of utterances: an annotator must check all preceding utterances to identify the one to which the current utterance is a reply. In this paper, we are the first to propose a~\textbf{zero-shot} dialogue disentanglement solution. Firstly, we train a model on a multi-participant response selection dataset harvested from the web which is not annotated; we then apply the trained model to perform zero-shot dialogue disentanglement. Without any labeled data, our model can achieve a cluster F1 score of 25. We also fine-tune the model using various amounts of labeled data. Experiments show that with only 10\% of the data, we achieve nearly the same performance of using the full dataset\footnote{Code is released at \url{https://github.com/chijames/zero_shot_dialogue_disentanglement}}.