GASTON: Graph-Aware Social Transformer for Online Networks

TL;DR

GASTON is a novel graph-aware transformer that integrates textual content and social context to improve detection of harmful online interactions, outperforming existing methods across multiple tasks.

Contribution

It introduces a contrastive pretraining strategy for community embeddings grounded in user patterns, enhancing context understanding in social media analysis.

Findings

GASTON outperforms state-of-the-art baselines in toxicity detection.

Community embeddings effectively distinguish different social groups.

Pretraining improves the model's ability to capture social norms.

Abstract

Online communities have become essential places for socialization and support, yet they also possess toxicity, echo chambers, and misinformation. Detecting this harmful content is difficult because the meaning of an online interaction stems from both what is written (textual content) and where it is posted (social norms). We propose GASTON (Graph-Aware Social Transformer for Online Networks), which learns text and user embeddings that are grounded in their local norms, providing the necessary context for downstream tasks. The heart of our solution is a contrastive initialization strategy that pretrains community embeddings based on user membership patterns, capturing a community's user base before processing any text. This allows GASTON to distinguish between communities (e.g., a support group vs. a hate group) based on who interacts there, even if they share similar vocabulary. Experiments on tasks such as stress detection, toxicity scoring, and norm violation demonstrate that the embeddings produced by GASTON outperform state-of-the-art baselines.