Cross-Lingual Task-Specific Representation Learning for Text Classification in Resource Poor Languages

TL;DR

This paper introduces a twin Bi-LSTM model with shared parameters and contrastive loss to improve text classification in resource-poor languages by leveraging resource-rich languages, showing significant performance gains.

Contribution

The paper proposes a novel cross-lingual representation learning approach using twin Bi-LSTM networks with shared parameters and contrastive loss for resource-poor language classification.

Findings

Model outperforms state-of-the-art methods in sentiment analysis and emoji prediction.

Significant improvements in classification accuracy for Hindi and Telugu.

Effective cross-lingual transfer between resource-rich and resource-poor languages.

Abstract

Neural network models have shown promising results for text classification. However, these solutions are limited by their dependence on the availability of annotated data. The prospect of leveraging resource-rich languages to enhance the text classification of resource-poor languages is fascinating. The performance on resource-poor languages can significantly improve if the resource availability constraints can be offset. To this end, we present a twin Bidirectional Long Short Term Memory (Bi-LSTM) network with shared parameters consolidated by a contrastive loss function (based on a similarity metric). The model learns the representation of resource-poor and resource-rich sentences in a common space by using the similarity between their assigned annotation tags. Hence, the model projects sentences with similar tags closer and those with different tags farther from each other. We evaluated our model on the classification tasks of sentiment analysis and emoji prediction for resource-poor languages - Hindi and Telugu and resource-rich languages - English and Spanish. Our model significantly outperforms the state-of-the-art approaches in both the tasks across all metrics.