Hate Personified: Investigating the role of LLMs in content moderation

TL;DR

This study explores how large language models' sensitivity to context, geography, and numerical cues affects hate detection, revealing variability based on persona, region, and numerical information, with implications for content moderation.

Contribution

It provides a comprehensive analysis of LLMs' responsiveness to different contextual cues in hate detection across multiple languages and datasets, highlighting key sensitivities and potential biases.

Findings

Persona attributes increase annotation variability.

Geographical signals improve regional alignment.

Numerical anchors influence model sensitivity.

Abstract

For subjective tasks such as hate detection, where people perceive hate differently, the Large Language Model's (LLM) ability to represent diverse groups is unclear. By including additional context in prompts, we comprehensively analyze LLM's sensitivity to geographical priming, persona attributes, and numerical information to assess how well the needs of various groups are reflected. Our findings on two LLMs, five languages, and six datasets reveal that mimicking persona-based attributes leads to annotation variability. Meanwhile, incorporating geographical signals leads to better regional alignment. We also find that the LLMs are sensitive to numerical anchors, indicating the ability to leverage community-based flagging efforts and exposure to adversaries. Our work provides preliminary guidelines and highlights the nuances of applying LLMs in culturally sensitive cases.