Measuring political sentiment on Twitter: factor-optimal design for multinomial inverse regression

TL;DR

This paper introduces a new algorithm for selecting representative Twitter posts to efficiently score political sentiment, utilizing a D-optimal design approach within multinomial inverse regression, demonstrated on 2.1 million tweets.

Contribution

It proposes a greedy, D-optimal sampling algorithm for text analysis and a novel technique for sentiment prediction using variable interactions in multinomial inverse regression.

Findings

Efficient sampling improves sentiment analysis accuracy.

The method effectively analyzes large-scale Twitter data.

New variable interaction technique enhances sentiment prediction.

Abstract

This article presents a short case study in text analysis: the scoring of Twitter posts for positive, negative, or neutral sentiment directed towards particular US politicians. The study requires selection of a sub-sample of representative posts for sentiment scoring, a common and costly aspect of sentiment mining. As a general contribution, our application is preceded by a proposed algorithm for maximizing sampling efficiency. In particular, we outline and illustrate greedy selection of documents to build designs that are D-optimal in a topic-factor decomposition of the original text. The strategy is applied to our motivating dataset of political posts, and we outline a new technique for predicting both generic and subject-specific document sentiment through use of variable interactions in multinomial inverse regression. Results are presented for analysis of 2.1 million Twitter posts around February 2012.