Identifying Actionable Messages on Social Media

TL;DR

This paper presents a supervised learning framework for large-scale, domain-aware classification of social media messages' actionability, analyzing extensive features across multiple languages and platforms.

Contribution

It introduces a comprehensive approach combining lexicons, feature analysis, and domain adaptation strategies for actionability detection on social media.

Findings

Achieved an aggregate F measure of 0.78 and accuracy of 0.74 across diverse datasets.

Analyzed over 25 text features for improved classification.

Demonstrated effectiveness across 75 companies and 35 languages.

Abstract

Text actionability detection is the problem of classifying user authored natural language text, according to whether it can be acted upon by a responding agent. In this paper, we propose a supervised learning framework for domain-aware, large-scale actionability classification of social media messages. We derive lexicons, perform an in-depth analysis for over 25 text based features, and explore strategies to handle domains that have limited training data. We apply these methods to over 46 million messages spanning 75 companies and 35 languages, from both Facebook and Twitter. The models achieve an aggregate population-weighted F measure of 0.78 and accuracy of 0.74, with values of over 0.9 in some cases.