Leveraging Language Identification to Enhance Code-Mixed Text Classification

TL;DR

This paper enhances BERT-based models for code-mixed Hindi-English text classification by incorporating language identification and augmentation techniques, significantly improving performance on various downstream tasks.

Contribution

It introduces a novel pipeline with language augmentation methods, including word-level interleaving and post-sentence placement, to improve code-mixed text classification.

Findings

Language augmentation improves model accuracy across tasks

Augmented models outperform vanilla BERT models

Effective on multiple downstream datasets

Abstract

The usage of more than one language in the same text is referred to as Code Mixed. It is evident that there is a growing degree of adaption of the use of code-mixed data, especially English with a regional language, on social media platforms. Existing deep-learning models do not take advantage of the implicit language information in the code-mixed text. Our study aims to improve BERT-based models performance on low-resource Code-Mixed Hindi-English Datasets by experimenting with language augmentation approaches. We propose a pipeline to improve code-mixed systems that comprise data preprocessing, word-level language identification, language augmentation, and model training on downstream tasks like sentiment analysis. For language augmentation in BERT models, we explore word-level interleaving and post-sentence placement of language information. We have examined the performance of vanilla BERT-based models and their code-mixed HingBERT counterparts on respective benchmark datasets, comparing their results with and without using word-level language information. The models were evaluated using metrics such as accuracy, precision, recall, and F1 score. Our findings show that the proposed language augmentation approaches work well across different BERT models. We demonstrate the importance of augmenting code-mixed text with language information on five different code-mixed Hindi-English downstream datasets based on sentiment analysis, hate speech detection, and emotion detection.