Benchmarking sentiment analysis methods for large-scale texts: A case for using continuum-scored words and word shift graphs

TL;DR

This paper evaluates six dictionary-based sentiment analysis methods on large-scale social media texts, emphasizing the importance of comprehensive lexicons and continuum scoring for reliable and meaningful sentiment estimation.

Contribution

It demonstrates that effective sentiment analysis requires large, frequency-weighted lexicons and continuous word scoring, providing a benchmark comparison of multiple methods.

Findings

Dictionary coverage and word scoring scale are crucial for reliability.

Continuum scoring improves sentiment measurement accuracy.

Benchmark results highlight the strengths and weaknesses of different methods.

Abstract

The emergence and global adoption of social media has rendered possible the real-time estimation of population-scale sentiment, bearing profound implications for our understanding of human behavior. Given the growing assortment of sentiment measuring instruments, comparisons between them are evidently required. Here, we perform detailed tests of 6 dictionary-based methods applied to 4 different corpora, and briefly examine a further 20 methods. We show that a dictionary-based method will only perform both reliably and meaningfully if (1) the dictionary covers a sufficiently large enough portion of a given text's lexicon when weighted by word usage frequency; and (2) words are scored on a continuous scale.