Challenges and Limitations with the Metrics Measuring the Complexity of Code-Mixed Text

TL;DR

This paper examines the limitations of current metrics used to measure the complexity of code-mixed text, highlighting challenges in accurately identifying and validating such multilingual content.

Contribution

It provides a critical analysis of existing code-mixing metrics, revealing their inherent limitations through examples from popular datasets.

Findings

Existing metrics have significant limitations in measuring code-mixed text complexity.

Current metrics often fail to accurately distinguish code-mixed from monolingual or noisy text.

The paper suggests the need for improved metrics for better code-mixed text analysis.

Abstract

Code-mixing is a frequent communication style among multilingual speakers where they mix words and phrases from two different languages in the same utterance of text or speech. Identifying and filtering code-mixed text is a challenging task due to its co-existence with monolingual and noisy text. Over the years, several code-mixing metrics have been extensively used to identify and validate code-mixed text quality. This paper demonstrates several inherent limitations of code-mixing metrics with examples from the already existing datasets that are popularly used across various experiments.