XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond

TL;DR

This paper introduces XLM-T, a multilingual language model trained on Twitter data across multiple languages, providing a new benchmark and datasets for sentiment analysis and beyond in social media NLP.

Contribution

The paper presents a new multilingual Twitter-specific language model, XLM-T, along with a strong baseline and unified sentiment datasets in eight languages for improved social media NLP tasks.

Findings

XLM-T outperforms existing models on multilingual Twitter sentiment analysis.

The model provides a versatile foundation for various multilingual social media NLP tasks.

The datasets enable standardized evaluation across multiple languages.

Abstract

Language models are ubiquitous in current NLP, and their multilingual capacity has recently attracted considerable attention. However, current analyses have almost exclusively focused on (multilingual variants of) standard benchmarks, and have relied on clean pre-training and task-specific corpora as multilingual signals. In this paper, we introduce XLM-T, a model to train and evaluate multilingual language models in Twitter. In this paper we provide: (1) a new strong multilingual baseline consisting of an XLM-R (Conneau et al. 2020) model pre-trained on millions of tweets in over thirty languages, alongside starter code to subsequently fine-tune on a target task; and (2) a set of unified sentiment analysis Twitter datasets in eight different languages and a XLM-T model fine-tuned on them.