UPB at SemEval-2020 Task 9: Identifying Sentiment in Code-Mixed Social Media Texts using Transformers and Multi-Task Learning

TL;DR

This paper describes systems using Transformer-based models and multi-task learning to identify sentiment in code-mixed social media texts, specifically for Hindi-English and Spanish-English, achieving competitive results in SemEval-2020.

Contribution

The paper introduces neural network approaches with pre-trained multilingual transformers for sentiment analysis in code-mixed languages, a challenging and less-explored area.

Findings

Multilingual BERT achieved 0.6850 F1-score on Hindi-English.

XLM-RoBERTa achieved 0.7064 F1-score on Spanish-English.

The systems ranked 16th and 17th in their respective tasks.

Abstract

Sentiment analysis is a process widely used in opinion mining campaigns conducted today. This phenomenon presents applications in a variety of fields, especially in collecting information related to the attitude or satisfaction of users concerning a particular subject. However, the task of managing such a process becomes noticeably more difficult when it is applied in cultures that tend to combine two languages in order to express ideas and thoughts. By interleaving words from two languages, the user can express with ease, but at the cost of making the text far less intelligible for those who are not familiar with this technique, but also for standard opinion mining algorithms. In this paper, we describe the systems developed by our team for SemEval-2020 Task 9 that aims to cover two well-known code-mixed languages: Hindi-English and Spanish-English. We intend to solve this issue by introducing a solution that takes advantage of several neural network approaches, as well as pre-trained word embeddings. Our approach (multlingual BERT) achieves promising performance on the Hindi-English task, with an average F1-score of 0.6850, registered on the competition leaderboard, ranking our team 16th out of 62 participants. For the Spanish-English task, we obtained an average F1-score of 0.7064 ranking our team 17th out of 29 participants by using another multilingual Transformer-based model, XLM-RoBERTa.