Deep Exogenous and Endogenous Influence Combination for Social Chatter Intensity Prediction

TL;DR

This paper introduces ChatterNet, a novel model that predicts social media discussion intensity by analyzing news and discussion streams without relying on user network data, capturing complex exogenous and endogenous influences.

Contribution

ChatterNet is the first framework to model and predict user engagement without using user network information, leveraging a time-evolving recurrent architecture for streaming news and discussions.

Findings

ChatterNet outperforms recent state-of-the-art models in engagement prediction.

It effectively captures the influence of external news on discussion volume.

The model provides insights across 43 different subreddits.

Abstract

Modeling user engagement dynamics on social media has compelling applications in user-persona detection and political discourse mining. Most existing approaches depend heavily on knowledge of the underlying user network. However, a large number of discussions happen on platforms that either lack any reliable social network or reveal only partially the inter-user ties (Reddit, Stackoverflow). Many approaches require observing a discussion for some considerable period before they can make useful predictions. In real-time streaming scenarios, observations incur costs. Lastly, most models do not capture complex interactions between exogenous events (such as news articles published externally) and in-network effects (such as follow-up discussions on Reddit) to determine engagement levels. To address the three limitations noted above, we propose a novel framework, ChatterNet, which, to our knowledge, is the first that can model and predict user engagement without considering the underlying user network. Given streams of timestamped news articles and discussions, the task is to observe the streams for a short period leading up to a time horizon, then predict chatter: the volume of discussions through a specified period after the horizon. ChatterNet processes text from news and discussions using a novel time-evolving recurrent network architecture that captures both temporal properties within news and discussions, as well as the influence of news on discussions. We report on extensive experiments using a two-month-long discussion corpus of Reddit, and a contemporaneous corpus of online news articles from the Common Crawl. ChatterNet shows considerable improvements beyond recent state-of-the-art models of engagement prediction. Detailed studies controlling observation and prediction windows, over 43 different subreddits, yield further useful insights.