HateBuffer: Safeguarding Content Moderators' Mental Well-Being through Hate Speech Content Modification

TL;DR

HateBuffer is a tool designed to protect content moderators from emotional harm by anonymizing and paraphrasing hate speech, showing promise as a supportive content moderation aid despite mixed measured effects.

Contribution

This paper introduces HateBuffer, a novel text modification system that anonymizes and paraphrases hate speech to support moderators' mental well-being.

Findings

Participants rated hate severity lower with HateBuffer.

HateBuffer did not significantly reduce emotional fatigue.

HateBuffer slightly improved moderation recall.

Abstract

Hate speech remains a persistent and unresolved challenge in online platforms. Content moderators, working on the front lines to review user-generated content and shield viewers from hate speech, often find themselves unprotected from the mental burden as they continuously engage with offensive language. To safeguard moderators' mental well-being, we designed HateBuffer, which anonymizes targets of hate speech, paraphrases offensive expressions into less offensive forms, and shows the original expressions when moderators opt to see them. Our user study with 80 participants consisted of a simulated hate speech moderation task set on a fictional news platform, followed by semi-structured interviews. Although participants rated the hate severity of comments lower while using HateBuffer, contrary to our expectations, they did not experience improved emotion or reduced fatigue compared with the control group. In interviews, however, participants described HateBuffer as an effective buffer against emotional contagion and the normalization of biased opinions in hate speech. Notably, HateBuffer did not compromise moderation accuracy and even contributed to a slight increase in recall. We explore possible explanations for the discrepancy between the perceived benefits of HateBuffer and its measured impact on mental well-being. We also underscore the promise of text-based content modification techniques as tools for a healthier content moderation environment.