To What Degree Can Language Borders Be Blurred In BERT-based Multilingual Spoken Language Understanding?

TL;DR

This paper investigates the transferability of BERT-based multilingual SLU models across languages, proposing an adversarial architecture to improve shared representations and reduce performance gaps.

Contribution

It introduces a novel BERT-based adversarial model architecture for multilingual SLU that enhances language sharing and reduces performance gaps.

Findings

Multilingual BERT-based SLU performs well across distant languages.

The proposed adversarial model narrows the performance gap.

Experimental results confirm improved language-shared representations.

Abstract

This paper addresses the question as to what degree a BERT-based multilingual Spoken Language Understanding (SLU) model can transfer knowledge across languages. Through experiments we will show that, although it works substantially well even on distant language groups, there is still a gap to the ideal multilingual performance. In addition, we propose a novel BERT-based adversarial model architecture to learn language-shared and language-specific representations for multilingual SLU. Our experimental results prove that the proposed model is capable of narrowing the gap to the ideal multilingual performance.