Palomino-Ochoa at SemEval-2020 Task 9: Robust System based on Transformer for Code-Mixed Sentiment Classification

TL;DR

This paper introduces a robust transfer learning system combining BERT and ULMFiT for code-mixed Spanish-English sentiment analysis, achieving competitive results in SemEval 2020 Task 9.

Contribution

The novel integration of BERT with ULMFiT for code-mixed sentiment classification improves performance and reproducibility in multilingual social media analysis.

Findings

Ranked 4th among 29 systems in SemEval 2020 Task 9

Achieved a weighted-F1 score of 0.755

System is easily reproducible with available source code

Abstract

We present a transfer learning system to perform a mixed Spanish-English sentiment classification task. Our proposal uses the state-of-the-art language model BERT and embed it within a ULMFiT transfer learning pipeline. This combination allows us to predict the polarity detection of code-mixed (English-Spanish) tweets. Thus, among 29 submitted systems, our approach (referred to as dplominop) is ranked 4th on the Sentimix Spanglish test set of SemEval 2020 Task 9. In fact, our system yields the weighted-F1 score value of 0.755 which can be easily reproduced -- the source code and implementation details are made available.