INSIGHT-1 at SemEval-2016 Task 4: Convolutional Neural Networks for Sentiment Classification and Quantification

TL;DR

This paper presents a CNN-based approach for Twitter sentiment analysis in SemEval-2016, achieving competitive results on certain subtasks using only pre-trained embeddings, and discusses limitations and potential improvements.

Contribution

Introduces a deep learning CNN model for sentiment classification and quantification on Twitter, highlighting its performance and limitations, and proposing future enhancements.

Findings

Achieved top-tier results in two-point sentiment classification and quantification.

Performed well on three-point classification but struggled with five-point tasks.

Identified model limitations in capturing negative and ordinal sentiment information.

Abstract

This paper describes our deep learning-based approach to sentiment analysis in Twitter as part of SemEval-2016 Task 4. We use a convolutional neural network to determine sentiment and participate in all subtasks, i.e. two-point, three-point, and five-point scale sentiment classification and two-point and five-point scale sentiment quantification. We achieve competitive results for two-point scale sentiment classification and quantification, ranking fifth and a close fourth (third and second by alternative metrics) respectively despite using only pre-trained embeddings that contain no sentiment information. We achieve good performance on three-point scale sentiment classification, ranking eighth out of 35, while performing poorly on five-point scale sentiment classification and quantification. An error analysis reveals that this is due to low expressiveness of the model to capture negative sentiment as well as an inability to take into account ordinal information. We propose improvements in order to address these and other issues.