How people talk about each other: Modeling Generalized Intergroup Bias and Emotion

TL;DR

This paper models generalized intergroup bias and emotion in social media text, introducing a new dataset and showing neural models outperform humans in predicting interpersonal group relationships and emotions.

Contribution

It introduces a novel dataset of annotated tweets for interpersonal emotion and IGR, and demonstrates improved modeling of bias through shared encoding of emotion and IGR.

Findings

Neural models outperform humans in IGR prediction.

Shared encoding improves performance on both IGR and emotion tasks.

Subtle emotional signals are indicative of interpersonal biases.

Abstract

Current studies of bias in NLP rely mainly on identifying (unwanted or negative) bias towards a specific demographic group. While this has led to progress recognizing and mitigating negative bias, and having a clear notion of the targeted group is necessary, it is not always practical. In this work we extrapolate to a broader notion of bias, rooted in social science and psychology literature. We move towards predicting interpersonal group relationship (IGR) - modeling the relationship between the speaker and the target in an utterance - using fine-grained interpersonal emotions as an anchor. We build and release a dataset of English tweets by US Congress members annotated for interpersonal emotion -- the first of its kind, and 'found supervision' for IGR labels; our analyses show that subtle emotional signals are indicative of different biases. While humans can perform better than chance at identifying IGR given an utterance, we show that neural models perform much better; furthermore, a shared encoding between IGR and interpersonal perceived emotion enabled performance gains in both tasks. Data and code for this paper are available at https://github.com/venkatasg/interpersonal-bias