Actionable and Political Text Classification using Word Embeddings and LSTM

TL;DR

This paper demonstrates the effectiveness of LSTM neural networks with word embeddings for classifying social media messages into actionable/non-actionable and political leanings, achieving high accuracy across multiple languages and applications.

Contribution

It introduces neural network models for multi-language actionability and political classification, outperforming traditional methods and deploying practical solutions.

Findings

LSTM with word embeddings achieves around 85-90% accuracy in actionability classification.

Political leaning classification reaches 87.57% accuracy.

Models are deployed in production and publicly available.

Abstract

In this work, we apply word embeddings and neural networks with Long Short-Term Memory (LSTM) to text classification problems, where the classification criteria are decided by the context of the application. We examine two applications in particular. The first is that of Actionability, where we build models to classify social media messages from customers of service providers as Actionable or Non-Actionable. We build models for over 30 different languages for actionability, and most of the models achieve accuracy around 85%, with some reaching over 90% accuracy. We also show that using LSTM neural networks with word embeddings vastly outperform traditional techniques. Second, we explore classification of messages with respect to political leaning, where social media messages are classified as Democratic or Republican. The model is able to classify messages with a high accuracy of 87.57%. As part of our experiments, we vary different hyperparameters of the neural networks, and report the effect of such variation on the accuracy. These actionability models have been deployed to production and help company agents provide customer support by prioritizing which messages to respond to. The model for political leaning has been opened and made available for wider use.