CSS: Combining Self-training and Self-supervised Learning for Few-shot Dialogue State Tracking

TL;DR

This paper introduces CSS, a novel framework for few-shot dialogue state tracking that combines self-training and self-supervised learning to effectively utilize unlabeled data and improve performance.

Contribution

CSS is the first framework to integrate self-training and contrastive self-supervised learning specifically for few-shot dialogue state tracking.

Findings

CSS achieves competitive results on MultiWOZ dataset.

Self-training improves pseudo-label quality for DST.

Contrastive learning enhances representation quality.

Abstract

Few-shot dialogue state tracking (DST) is a realistic problem that trains the DST model with limited labeled data. Existing few-shot methods mainly transfer knowledge learned from external labeled dialogue data (e.g., from question answering, dialogue summarization, machine reading comprehension tasks, etc.) into DST, whereas collecting a large amount of external labeled data is laborious, and the external data may not effectively contribute to the DST-specific task. In this paper, we propose a few-shot DST framework called CSS, which Combines Self-training and Self-supervised learning methods. The unlabeled data of the DST task is incorporated into the self-training iterations, where the pseudo labels are predicted by a DST model trained on limited labeled data in advance. Besides, a contrastive self-supervised method is used to learn better representations, where the data is augmented by the dropout operation to train the model. Experimental results on the MultiWOZ dataset show that our proposed CSS achieves competitive performance in several few-shot scenarios.