Fairness across Network Positions in Cyberbullying Detection Algorithms

TL;DR

This paper investigates bias in cyberbullying detection algorithms related to users' network positions on social media and demonstrates methods to improve fairness across these roles.

Contribution

It empirically quantifies how detection performance varies with network centrality and applies fairness techniques to mitigate this bias.

Findings

Detection disparity exists based on network centrality.

Equalized Odds can reduce bias in detection performance.

Fairness improvements lead to more equitable cyberbullying detection.

Abstract

Cyberbullying, which often has a deeply negative impact on the victim, has grown as a serious issue in Online Social Networks. Recently, researchers have created automated machine learning algorithms to detect Cyberbullying using social and textual features. However, the very algorithms that are intended to fight off one threat (cyberbullying) may inadvertently be falling prey to another important threat (bias of the automatic detection algorithms). This is exacerbated by the fact that while the current literature on algorithmic fairness has multiple empirical results, metrics, and algorithms for countering bias across immediately observable demographic characteristics (e.g. age, race, gender), there have been no efforts at empirically quantifying the variation in algorithmic performance based on the network role or position of individuals. We audit an existing cyberbullying algorithm using Twitter data for disparity in detection performance based on the network centrality of the potential victim and then demonstrate how this disparity can be countered using an Equalized Odds post-processing technique. The results pave way for more accurate and fair cyberbullying detection algorithms.