Blind Men and the Elephant: Detecting Evolving Groups In Social News

TL;DR

This paper introduces an unsupervised, multi-layered approach combining graph theory and text analysis to detect and summarize the evolution of social groups over time, demonstrated on political discourse data.

Contribution

It presents a novel methodology for analyzing evolving online groups using community detection and content summarization, with a new entropy-based evaluation approach.

Findings

Identified long-term political group coexistence and shifts due to external events.

Demonstrated effectiveness on four years of social news data.

Proposed entropy measures as a new evaluation method for group coherence.

Abstract

We propose an automated and unsupervised methodology for a novel summarization of group behavior based on content preference. We show that graph theoretical community evolution (based on similarity of user preference for content) is effective in indexing these dynamics. Combined with text analysis that targets automatically-identified representative content for each community, our method produces a novel multi-layered representation of evolving group behavior. We demonstrate this methodology in the context of political discourse on a social news site with data that spans more than four years and find coexisting political leanings over extended periods and a disruptive external event that lead to a significant reorganization of existing patterns. Finally, where there exists no ground truth, we propose a new evaluation approach by using entropy measures as evidence of coherence along the evolution path of these groups. This methodology is valuable to designers and managers of online forums in need of granular analytics of user activity, as well as to researchers in social and political sciences who wish to extend their inquiries to large-scale data available on the web.