Universal Cross-Lingual Text Classification

TL;DR

This paper introduces a universal cross-lingual text classification model that leverages multilingual data and a strong SBERT base to improve label and language coverage, especially for low-resource languages.

Contribution

It proposes a novel training strategy using multilingual data blending with SBERT to create an adaptable, universal classifier across multiple languages.

Findings

Enhanced label coverage across languages

Improved classification accuracy in low-resource languages

Effective cross-lingual transfer demonstrated

Abstract

Text classification, an integral task in natural language processing, involves the automatic categorization of text into predefined classes. Creating supervised labeled datasets for low-resource languages poses a considerable challenge. Unlocking the language potential of low-resource languages requires robust datasets with supervised labels. However, such datasets are scarce, and the label space is often limited. In our pursuit to address this gap, we aim to optimize existing labels/datasets in different languages. This research proposes a novel perspective on Universal Cross-Lingual Text Classification, leveraging a unified model across languages. Our approach involves blending supervised data from different languages during training to create a universal model. The supervised data for a target classification task might come from different languages covering different labels. The primary goal is to enhance label and language coverage, aiming for a label set that represents a union of labels from various languages. We propose the usage of a strong multilingual SBERT as our base model, making our novel training strategy feasible. This strategy contributes to the adaptability and effectiveness of the model in cross-lingual language transfer scenarios, where it can categorize text in languages not encountered during training. Thus, the paper delves into the intricacies of cross-lingual text classification, with a particular focus on its application for low-resource languages, exploring methodologies and implications for the development of a robust and adaptable universal cross-lingual model.