Hope vs. Hate: Understanding User Interactions with LGBTQ+ News Content in Mainstream US News Media through the Lens of Hope Speech

TL;DR

This study analyzes user comments on US news videos about LGBTQ+ topics, developing a hope speech classifier, creating a labeled dataset, and examining how political beliefs influence content perception and model agreement.

Contribution

It introduces a hope speech classifier, provides a diverse annotated dataset, and investigates the impact of political beliefs on content interpretation and model performance.

Findings

Strong link between political beliefs and content ratings.

Models trained on individual beliefs show disagreement.

Zero-shot LLMs align more with liberal raters.

Abstract

This paper makes three contributions. First, via a substantial corpus of 1,419,047 comments posted on 3,161 YouTube news videos of major US cable news outlets, we analyze how users engage with LGBTQ+ news content. Our analyses focus both on positive and negative content. In particular, we construct a fine-grained hope speech classifier that detects positive (hope speech), negative, neutral, and irrelevant content. Second, in consultation with a public health expert specializing on LGBTQ+ health, we conduct an annotation study with a balanced and diverse political representation and release a dataset of 3,750 instances with fine-grained labels and detailed annotator demographic information. Finally, beyond providing a vital resource for the LGBTQ+ community, our annotation study and subsequent in-the-wild assessments reveal (1) strong association between rater political beliefs and how they rate content relevant to a marginalized community; (2) models trained on individual political beliefs exhibit considerable in-the-wild disagreement; and (3) zero-shot large language models (LLMs) align more with liberal raters.