Toxic Bias: Perspective API Misreads German as More Toxic
Gianluca Nogara, Francesco Pierri, Stefano Cresci, Luca Luceri, Petter, T\"ornberg, Silvia Giordano

TL;DR
This paper reveals that Google's Perspective API exhibits a language bias, systematically overestimating toxicity in German social media content compared to other languages, impacting research and moderation practices.
Contribution
It uncovers intrinsic language bias in Perspective API's multilingual model, demonstrating higher toxicity scores for German content across datasets and translations.
Findings
German toxicity scores are significantly higher than in other languages.
Using German leads to four times more moderation actions than English.
Bias persists across various datasets, topics, and translations.
Abstract
Proprietary public APIs play a crucial and growing role as research tools among social scientists. Among such APIs, Google's machine learning-based Perspective API is extensively utilized for assessing the toxicity of social media messages, providing both an important resource for researchers and automatic content moderation. However, this paper exposes an important bias in Perspective API concerning German language text. Through an in-depth examination of several datasets, we uncover intrinsic language biases within the multilingual model of Perspective API. We find that the toxicity assessment of German content produces significantly higher toxicity levels than other languages. This finding is robust across various translations, topics, and data sources, and has significant consequences for both research and moderation strategies that rely on Perspective API. For instance, we show…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHate Speech and Cyberbullying Detection
