HC$^2$L: Hybrid and Cooperative Contrastive Learning for Cross-lingual Spoken Language Understanding

TL;DR

This paper introduces HC$^2$L, a novel contrastive learning framework that combines supervised and unsupervised methods across multiple languages to improve zero-shot cross-lingual spoken language understanding.

Contribution

It proposes a comprehensive hybrid and cooperative contrastive learning approach leveraging label information to enhance semantic alignment in cross-lingual SLU.

Findings

Achieves state-of-the-art results on 9 languages.

Improves semantic alignment by integrating multiple contrastive learning mechanisms.

Demonstrates consistent performance gains over previous models.

Abstract

State-of-the-art model for zero-shot cross-lingual spoken language understanding performs cross-lingual unsupervised contrastive learning to achieve the label-agnostic semantic alignment between each utterance and its code-switched data. However, it ignores the precious intent/slot labels, whose label information is promising to help capture the label-aware semantics structure and then leverage supervised contrastive learning to improve both source and target languages' semantics. In this paper, we propose Hybrid and Cooperative Contrastive Learning to address this problem. Apart from cross-lingual unsupervised contrastive learning, we design a holistic approach that exploits source language supervised contrastive learning, cross-lingual supervised contrastive learning and multilingual supervised contrastive learning to perform label-aware semantics alignments in a comprehensive manner. Each kind of supervised contrastive learning mechanism includes both single-task and joint-task scenarios. In our model, one contrastive learning mechanism's input is enhanced by others. Thus the total four contrastive learning mechanisms are cooperative to learn more consistent and discriminative representations in the virtuous cycle during the training process. Experiments show that our model obtains consistent improvements over 9 languages, achieving new state-of-the-art performance.