On the Effect of Word Order on Cross-lingual Sentiment Analysis

TL;DR

This paper investigates how reordering words as a preprocessing step affects cross-lingual sentiment analysis, revealing that local and global reordering differently benefit CNNs and RNNs across language pairs.

Contribution

It introduces the use of reordering as a preprocessing step to improve cross-lingual sentiment classification and compares its effects on CNN and RNN models.

Findings

Reordering improves cross-lingual sentiment analysis performance.

CNNs are more sensitive to local reordering.

RNNs benefit more from global reordering.

Abstract

Current state-of-the-art models for sentiment analysis make use of word order either explicitly by pre-training on a language modeling objective or implicitly by using recurrent neural networks (RNNs) or convolutional networks (CNNs). This is a problem for cross-lingual models that use bilingual embeddings as features, as the difference in word order between source and target languages is not resolved. In this work, we explore reordering as a pre-processing step for sentence-level cross-lingual sentiment classification with two language combinations (English-Spanish, English-Catalan). We find that while reordering helps both models, CNNS are more sensitive to local reorderings, while global reordering benefits RNNs.