Discovering Emerging Topics in Social Streams via Link Anomaly Detection

TL;DR

This paper introduces a novel method for detecting emerging topics in social networks by analyzing link anomalies, specifically user mention behaviors, which can identify new topics earlier than traditional text-based methods.

Contribution

It proposes a probabilistic model of user mention behavior and combines it with change-point detection techniques to identify emerging topics from social network links.

Findings

Detects new topics using mention anomalies in social networks.

Performs as well as or earlier than traditional keyword-based methods.

Effective in scenarios with ill-defined keywords.

Abstract

Detection of emerging topics are now receiving renewed interest motivated by the rapid growth of social networks. Conventional term-frequency-based approaches may not be appropriate in this context, because the information exchanged are not only texts but also images, URLs, and videos. We focus on the social aspects of theses networks. That is, the links between users that are generated dynamically intentionally or unintentionally through replies, mentions, and retweets. We propose a probability model of the mentioning behaviour of a social network user, and propose to detect the emergence of a new topic from the anomaly measured through the model. We combine the proposed mention anomaly score with a recently proposed change-point detection technique based on the Sequentially Discounting Normalized Maximum Likelihood (SDNML), or with Kleinberg's burst model. Aggregating anomaly scores from hundreds of users, we show that we can detect emerging topics only based on the reply/mention relationships in social network posts. We demonstrate our technique in a number of real data sets we gathered from Twitter. The experiments show that the proposed mention-anomaly-based approaches can detect new topics at least as early as the conventional term-frequency-based approach, and sometimes much earlier when the keyword is ill-defined.