Multilingual Stance Detection: The Catalonia Independence Corpus

TL;DR

This paper introduces a new balanced multilingual Twitter dataset for stance detection related to Catalonia's independence, and demonstrates improved results with supervised models, advancing multilingual and cross-lingual stance detection research.

Contribution

It presents a new balanced multilingual dataset for stance detection in Catalan and Spanish, along with semi-automatic annotation methods and state-of-the-art experimental results.

Findings

New balanced dataset improves stance detection research.

Supervised models achieve state-of-the-art results.

Cross-lingual experiments demonstrate the dataset's utility.

Abstract

Stance detection aims to determine the attitude of a given text with respect to a specific topic or claim. While stance detection has been fairly well researched in the last years, most the work has been focused on English. This is mainly due to the relative lack of annotated data in other languages. The TW-10 Referendum Dataset released at IberEval 2018 is a previous effort to provide multilingual stance-annotated data in Catalan and Spanish. Unfortunately, the TW-10 Catalan subset is extremely imbalanced. This paper addresses these issues by presenting a new multilingual dataset for stance detection in Twitter for the Catalan and Spanish languages, with the aim of facilitating research on stance detection in multilingual and cross-lingual settings. The dataset is annotated with stance towards one topic, namely, the independence of Catalonia. We also provide a semi-automatic method to annotate the dataset based on a categorization of Twitter users. We experiment on the new corpus with a number of supervised approaches, including linear classifiers and deep learning methods. Comparison of our new corpus with the with the TW-1O dataset shows both the benefits and potential of a well balanced corpus for multilingual and cross-lingual research on stance detection. Finally, we establish new state-of-the-art results on the TW-10 dataset, both for Catalan and Spanish.