FiSSA at SemEval-2020 Task 9: Fine-tuned For Feelings

TL;DR

This paper evaluates various Transformer-based models for sentiment analysis on Spanish-English code-mixed social media data, introducing a two-step fine-tuning approach that improves performance.

Contribution

The paper compares monolingual and multilingual models and proposes a novel two-step fine-tuning method for better sentiment classification.

Findings

XLM-RoBERTa achieved the highest weighted F1-score of 0.537 on development data.

Two-step fine-tuning outperforms standard fine-tuning.

Team ranked tenth overall in SemEval-2020 Task 9.

Abstract

In this paper, we present our approach for sentiment classification on Spanish-English code-mixed social media data in the SemEval-2020 Task 9. We investigate performance of various pre-trained Transformer models by using different fine-tuning strategies. We explore both monolingual and multilingual models with the standard fine-tuning method. Additionally, we propose a custom model that we fine-tune in two steps: once with a language modeling objective, and once with a task-specific objective. Although two-step fine-tuning improves sentiment classification performance over the base model, the large multilingual XLM-RoBERTa model achieves best weighted F1-score with 0.537 on development data and 0.739 on test data. With this score, our team jupitter placed tenth overall in the competition.