Toxic Bias: Perspective API Misreads German as More Toxic

TL;DR

This paper reveals that Google's Perspective API exhibits a language bias, systematically overestimating toxicity in German social media content compared to other languages, impacting research and moderation practices.

Contribution

It uncovers intrinsic language bias in Perspective API's multilingual model, demonstrating higher toxicity scores for German content across datasets and translations.

Findings

German toxicity scores are significantly higher than in other languages.

Using German leads to four times more moderation actions than English.

Bias persists across various datasets, topics, and translations.

Abstract

Proprietary public APIs play a crucial and growing role as research tools among social scientists. Among such APIs, Google's machine learning-based Perspective API is extensively utilized for assessing the toxicity of social media messages, providing both an important resource for researchers and automatic content moderation. However, this paper exposes an important bias in Perspective API concerning German language text. Through an in-depth examination of several datasets, we uncover intrinsic language biases within the multilingual model of Perspective API. We find that the toxicity assessment of German content produces significantly higher toxicity levels than other languages. This finding is robust across various translations, topics, and data sources, and has significant consequences for both research and moderation strategies that rely on Perspective API. For instance, we show that, on average, four times more tweets and users would be moderated when using the German language compared to their English translation. Our findings point to broader risks associated with the widespread use of proprietary APIs within the computational social sciences.