Code-Switching Language Modeling using Syntax-Aware Multi-Task Learning

TL;DR

This paper presents a syntax-aware multi-task learning approach for code-switching language modeling that leverages linguistic features to improve prediction accuracy in low-resource scenarios.

Contribution

It introduces a multi-task model that jointly learns language modeling and POS tagging to better identify switch points and enhance prediction in code-switched data.

Findings

Outperforms standard LSTM models with 9.7% and 7.4% perplexity reduction.

Leverages syntax sharing to address low-resource data issues.

Joint learning improves switch point detection and language modeling accuracy.

Abstract

Lack of text data has been the major issue on code-switching language modeling. In this paper, we introduce multi-task learning based language model which shares syntax representation of languages to leverage linguistic information and tackle the low resource data issue. Our model jointly learns both language modeling and Part-of-Speech tagging on code-switched utterances. In this way, the model is able to identify the location of code-switching points and improves the prediction of next word. Our approach outperforms standard LSTM based language model, with an improvement of 9.7% and 7.4% in perplexity on SEAME Phase I and Phase II dataset respectively.