SemEval-2014 Task 9: Sentiment Analysis in Twitter

TL;DR

This paper presents the SemEval-2014 Twitter sentiment analysis task, including new test sets and results from multiple teams, advancing the evaluation of sentiment detection in social media data.

Contribution

It introduces new test datasets for sarcastic tweets and LiveJournal sentences, and reports on the participation and performance in the 2014 sentiment analysis challenge.

Findings

Highest F1-score on regular tweets: 86.63 by NRC-Canada

Subtask B top F1-score: 70.96 by TeamX

Increased participation with 46 teams and 77 submissions

Abstract

We describe the Sentiment Analysis in Twitter task, ran as part of SemEval-2014. It is a continuation of the last year's task that ran successfully as part of SemEval-2013. As in 2013, this was the most popular SemEval task; a total of 46 teams contributed 27 submissions for subtask A (21 teams) and 50 submissions for subtask B (44 teams). This year, we introduced three new test sets: (i) regular tweets, (ii) sarcastic tweets, and (iii) LiveJournal sentences. We further tested on (iv) 2013 tweets, and (v) 2013 SMS messages. The highest F1-score on (i) was achieved by NRC-Canada at 86.63 for subtask A and by TeamX at 70.96 for subtask B.