Leveraging Large-scale Multimedia Datasets to Refine Content Moderation Models

TL;DR

This paper presents CM-Refinery, a framework that uses large-scale multimedia datasets to automatically enhance content moderation models by generating hard examples, reducing human annotation effort, and improving accuracy on NSFW and disturbing content detection tasks.

Contribution

The paper introduces CM-Refinery, a novel method leveraging large datasets and diversity criteria to automatically extend training data, improving content moderation models with minimal human involvement.

Findings

Achieved 1.32% and 1.94% accuracy improvements on benchmark datasets.

Automatically annotated 92.54% of disturbing content data, reducing human effort.

Enhanced model generalization through diversity-based data collection.

Abstract

The sheer volume of online user-generated content has rendered content moderation technologies essential in order to protect digital platform audiences from content that may cause anxiety, worry, or concern. Despite the efforts towards developing automated solutions to tackle this problem, creating accurate models remains challenging due to the lack of adequate task-specific training data. The fact that manually annotating such data is a highly demanding procedure that could severely affect the annotators' emotional well-being is directly related to the latter limitation. In this paper, we propose the CM-Refinery framework that leverages large-scale multimedia datasets to automatically extend initial training datasets with hard examples that can refine content moderation models, while significantly reducing the involvement of human annotators. We apply our method on two model adaptation strategies designed with respect to the different challenges observed while collecting data, i.e. lack of (i) task-specific negative data or (ii) both positive and negative data. Additionally, we introduce a diversity criterion applied to the data collection process that further enhances the generalization performance of the refined models. The proposed method is evaluated on the Not Safe for Work (NSFW) and disturbing content detection tasks on benchmark datasets achieving 1.32% and 1.94% accuracy improvements compared to the state of the art, respectively. Finally, it significantly reduces human involvement, as 92.54% of data are automatically annotated in case of disturbing content while no human intervention is required for the NSFW task.