Detecting Harmful Content On Online Platforms: What Platforms Need Vs. Where Research Efforts Go

TL;DR

This paper surveys the gap between the types of harmful online content platforms aim to control and the focus of current research efforts, highlighting areas for future investigation.

Contribution

It provides a comprehensive analysis of existing detection methods and moderation policies, identifying mismatches and suggesting future research directions.

Findings

Research often focuses on narrow harmful content types

Platforms seek to curb diverse harmful content

Significant gaps exist between platform needs and research efforts

Abstract

The proliferation of harmful content on online platforms is a major societal problem, which comes in many different forms including hate speech, offensive language, bullying and harassment, misinformation, spam, violence, graphic content, sexual abuse, self harm, and many other. Online platforms seek to moderate such content to limit societal harm, to comply with legislation, and to create a more inclusive environment for their users. Researchers have developed different methods for automatically detecting harmful content, often focusing on specific sub-problems or on narrow communities, as what is considered harmful often depends on the platform and on the context. We argue that there is currently a dichotomy between what types of harmful content online platforms seek to curb, and what research efforts there are to automatically detect such content. We thus survey existing methods as well as content moderation policies by online platforms in this light and we suggest directions for future work.