Comparing and Combining Sentiment Analysis Methods

TL;DR

This paper compares eight popular sentiment analysis methods in social media, introduces a new combined approach with improved coverage, and provides a web service for method comparison.

Contribution

It offers a systematic comparison of sentiment analysis methods, introduces a novel combination approach, and provides an accessible web API for analysis and comparison.

Findings

The new combined method achieves the highest coverage.

The comparison reveals strengths and weaknesses of existing methods.

The iFeel web service enables easy access to multiple sentiment analysis results.

Abstract

Several messages express opinions about events, products, and services, political views or even their author's emotional state and mood. Sentiment analysis has been used in several applications including analysis of the repercussions of events in social networks, analysis of opinions about products and services, and simply to better understand aspects of social communication in Online Social Networks (OSNs). There are multiple methods for measuring sentiments, including lexical-based approaches and supervised machine learning methods. Despite the wide use and popularity of some methods, it is unclear which method is better for identifying the polarity (i.e., positive or negative) of a message as the current literature does not provide a method of comparison among existing methods. Such a comparison is crucial for understanding the potential limitations, advantages, and disadvantages of popular methods in analyzing the content of OSNs messages. Our study aims at filling this gap by presenting comparisons of eight popular sentiment analysis methods in terms of coverage (i.e., the fraction of messages whose sentiment is identified) and agreement (i.e., the fraction of identified sentiments that are in tune with ground truth). We develop a new method that combines existing approaches, providing the best coverage results and competitive agreement. We also present a free Web service called iFeel, which provides an open API for accessing and comparing results across different sentiment methods for a given text.