Regionalized models for Spanish language variations based on Twitter

TL;DR

This paper develops regionalized Spanish language models using Twitter data from 26 countries, enhancing understanding of local language variations for improved regional NLP tasks.

Contribution

Introduces regionalized Spanish language resources including embeddings, BERT models, and corpora, with comprehensive regional comparison and application examples.

Findings

Regional language models improve regional NLP task performance.

Lexical and semantic differences vary significantly across regions.

Regional resources enable more accurate message classification.

Abstract

Spanish is one of the most spoken languages in the globe, but not necessarily Spanish is written and spoken in the same way in different countries. Understanding local language variations can help to improve model performances on regional tasks, both understanding local structures and also improving the message's content. For instance, think about a machine learning engineer who automatizes some language classification task on a particular region or a social scientist trying to understand a regional event with echoes on social media; both can take advantage of dialect-based language models to understand what is happening with more contextual information hence more precision. This manuscript presents and describes a set of regionalized resources for the Spanish language built on four-year Twitter public messages geotagged in 26 Spanish-speaking countries. We introduce word embeddings based on FastText, language models based on BERT, and per-region sample corpora. We also provide a broad comparison among regions covering lexical and semantical similarities; as well as examples of using regional resources on message classification tasks.