Two-Level Supervised Contrastive Learning for Response Selection in Multi-Turn Dialogue

TL;DR

This paper introduces a novel two-level supervised contrastive learning approach for response selection in multi-turn dialogue systems, improving representation separation and matching accuracy over existing methods.

Contribution

It proposes a new two-level supervised contrastive learning method utilizing sentence shuffling and re-ordering techniques for better response selection.

Findings

Significant performance improvements on three benchmark datasets.

Outperforms existing contrastive learning baselines and state-of-the-art methods.

Effective use of sentence shuffling and re-ordering techniques in contrastive learning.

Abstract

Selecting an appropriate response from many candidates given the utterances in a multi-turn dialogue is the key problem for a retrieval-based dialogue system. Existing work formalizes the task as matching between the utterances and a candidate and uses the cross-entropy loss in learning of the model. This paper applies contrastive learning to the problem by using the supervised contrastive loss. In this way, the learned representations of positive examples and representations of negative examples can be more distantly separated in the embedding space, and the performance of matching can be enhanced. We further develop a new method for supervised contrastive learning, referred to as two-level supervised contrastive learning, and employ the method in response selection in multi-turn dialogue. Our method exploits two techniques: sentence token shuffling (STS) and sentence re-ordering (SR) for supervised contrastive learning. Experimental results on three benchmark datasets demonstrate that the proposed method significantly outperforms the contrastive learning baseline and the state-of-the-art methods for the task.