SemEval-2013 Task 2: Sentiment Analysis in Twitter

TL;DR

This paper introduces SemEval-2013 Task 2, a benchmark for sentiment analysis in Twitter, providing labeled datasets and evaluating multiple approaches to advance research in social media sentiment classification.

Contribution

It presents a large, crowdsourced labeled Twitter dataset for sentiment analysis and organizes a shared task with multiple submissions to foster progress.

Findings

High-performing models achieved 88.9% F1 on expression-level sentiment

Significant research interest with 149 submissions from 44 teams

Datasets released publicly for future research

Abstract

In recent years, sentiment analysis in social media has attracted a lot of research interest and has been used for a number of applications. Unfortunately, research has been hindered by the lack of suitable datasets, complicating the comparison between approaches. To address this issue, we have proposed SemEval-2013 Task 2: Sentiment Analysis in Twitter, which included two subtasks: A, an expression-level subtask, and B, a message-level subtask. We used crowdsourcing on Amazon Mechanical Turk to label a large Twitter training dataset along with additional test sets of Twitter and SMS messages for both subtasks. All datasets used in the evaluation are released to the research community. The task attracted significant interest and a total of 149 submissions from 44 teams. The best-performing team achieved an F1 of 88.9% and 69% for subtasks A and B, respectively.