"Not in My Backyard": LLMs Uncover Online and Offline Social Biases Against Homelessness

TL;DR

This paper investigates online and offline social biases against homelessness, demonstrating that data quantity significantly impacts classifier performance and revealing prevalent negative biases, especially on Reddit, with implications for policy and social perception.

Contribution

It introduces a new multi-domain dataset and shows how GPT-4.1-generated pseudo-labels improve bias detection models, emphasizing data quantity over model size for effective bias classification.

Findings

Negative bias against PEH is widespread online and offline.

GPT-4.1 pseudo-labeling enhances classifier performance.

Small models can approach GPT-4.1 accuracy with sufficient data.

Abstract

Homelessness is a persistent social challenge, impacting millions worldwide. Over 876,000 people experienced homelessness (PEH) in the U.S. in 2025. Social bias is a significant barrier to alleviation, shaping public perception and influencing policymaking. Given that online textual media and offline city council discourse reflect and influence part of public opinion, it provides valuable insights to identify and track social biases against PEH. We present a new, manually-annotated multi-domain dataset compiled from Reddit, X (formerly Twitter), news articles, and city council meeting minutes across ten U.S. cities. Our 16-category multi-label taxonomy creates a challenging long-tail classification problem: some categories appear in less than 1% of samples, while others exceed 70%. We find that small human-annotated datasets (1,702 samples) are insufficient for training effective classifiers, whether used to fine-tune encoder models or as few-shot examples for LLMs. To address this, we use GPT-4.1 to generate pseudo-labels on a larger unlabeled corpus. Training on this expanded dataset enables even small encoder models (ModernBERT, 150M parameters) to achieve 35.23 macro-F1, approaching GPT-4.1's 41.57. This demonstrates that \textbf{data quantity matters more than model size}, enabling low-cost, privacy-preserving deployment without relying on commercial APIs. Our results reveal that negative bias against PEH is prevalent both offline and online (especially on Reddit), with "not in my backyard" narratives showing the highest engagement. These findings uncover a type of ostracism that directly impacts poverty-reduction policymaking and provide actionable insights for practitioners addressing homelessness.