Twitter Watch: Leveraging Social Media to Monitor and Predict Collective-Efficacy of Neighborhoods

TL;DR

This paper explores using social media data, specifically Twitter, to predict neighborhood collective efficacy, offering a cost-effective alternative to traditional survey methods with promising accuracy.

Contribution

It introduces a novel approach combining linguistic and topological features from Twitter data with a pairwise learning to rank model to estimate neighborhood collective efficacy.

Findings

Achieved 0.77 Kendall tau-x ranking agreement with ground truth

Outperformed traditional baselines by up to 37%

Demonstrated social media as a viable proxy for survey-based measures

Abstract

Sociologists associate the spatial variation of crime within an urban setting, with the concept of collective efficacy. The collective efficacy of a neighborhood is defined as social cohesion among neighbors combined with their willingness to intervene on behalf of the common good. Sociologists measure collective efficacy by conducting survey studies designed to measure individuals' perception of their community. In this work, we employ the curated data from a survey study (ground truth) and examine the effectiveness of substituting costly survey questionnaires with proxies derived from social media. We enrich a corpus of tweets mentioning a local venue with several linguistic and topological features. We then propose a pairwise learning to rank model with the goal of identifying a ranking of neighborhoods that is similar to the ranking obtained from the ground truth collective efficacy values. In our experiments, we find that our generated ranking of neighborhoods achieves 0.77 Kendall tau-x ranking agreement with the ground truth ranking. Overall, our results are up to 37% better than traditional baselines.