Residualized Factor Adaptation for Community Social Media Prediction Tasks

TL;DR

This paper introduces residualized factor adaptation, a new method that improves community outcome predictions from social media language by integrating and adapting to socio-demographic attributes, outperforming previous approaches.

Contribution

The paper proposes a novel residualized factor adaptation approach that effectively incorporates community socio-demographic attributes into linguistic prediction models.

Findings

Significantly improves 4 out of 5 community outcome predictions

Uses eleven demographic and socioeconomic attributes

Applies to health, psychology, and economics tasks

Abstract

Predictive models over social media language have shown promise in capturing community outcomes, but approaches thus far largely neglect the socio-demographic context (e.g. age, education rates, race) of the community from which the language originates. For example, it may be inaccurate to assume people in Mobile, Alabama, where the population is relatively older, will use words the same way as those from San Francisco, where the median age is younger with a higher rate of college education. In this paper, we present residualized factor adaptation, a novel approach to community prediction tasks which both (a) effectively integrates community attributes, as well as (b) adapts linguistic features to community attributes (factors). We use eleven demographic and socioeconomic attributes, and evaluate our approach over five different community-level predictive tasks, spanning health (heart disease mortality, percent fair/poor health), psychology (life satisfaction), and economics (percent housing price increase, foreclosure rate). Our evaluation shows that residualized factor adaptation significantly improves 4 out of 5 community-level outcome predictions over prior state-of-the-art for incorporating socio-demographic contexts.