Measuring Praise and Criticism: Inference of Semantic Orientation from Association

TL;DR

This paper presents a method to automatically determine the positive or negative semantic orientation of words by analyzing their statistical associations with known positive and negative words, useful for various text analysis applications.

Contribution

It introduces a novel approach using statistical association measures (PMI and LSA) to infer semantic orientation, with experimental validation on a large labeled word dataset.

Findings

Achieved 82.8% accuracy on the full test set.

Accuracy exceeds 95% when abstaining from classifying mild words.

Demonstrated effectiveness across various parts of speech.

Abstract

The evaluative character of a word is called its semantic orientation. Positive semantic orientation indicates praise (e.g., "honest", "intrepid") and negative semantic orientation indicates criticism (e.g., "disturbing", "superfluous"). Semantic orientation varies in both direction (positive or negative) and degree (mild to strong). An automated system for measuring semantic orientation would have application in text classification, text filtering, tracking opinions in online discussions, analysis of survey responses, and automated chat systems (chatbots). This paper introduces a method for inferring the semantic orientation of a word from its statistical association with a set of positive and negative paradigm words. Two instances of this approach are evaluated, based on two different statistical measures of word association: pointwise mutual information (PMI) and latent semantic analysis (LSA). The method is experimentally tested with 3,596 words (including adjectives, adverbs, nouns, and verbs) that have been manually labeled positive (1,614 words) and negative (1,982 words). The method attains an accuracy of 82.8% on the full test set, but the accuracy rises above 95% when the algorithm is allowed to abstain from classifying mild words.