DeMod: A Holistic Tool with Explainable Detection and Personalized Modification for Toxicity Censorship

TL;DR

DeMod is a comprehensive, user-friendly tool that combines explainable toxicity detection with personalized content modification, addressing complex censorship needs beyond mere detection.

Contribution

This paper introduces DeMod, a novel ChatGPT-based system that integrates explainable detection and personalized content modification for toxicity censorship.

Findings

DeMod achieves high accuracy in toxicity detection.

Users find DeMod's explanations and modifications helpful.

The tool demonstrates ease of use and functional richness.

Abstract

Although there have been automated approaches and tools supporting toxicity censorship for social posts, most of them focus on detection. Toxicity censorship is a complex process, wherein detection is just an initial task and a user can have further needs such as rationale understanding and content modification. For this problem, we conduct a needfinding study to investigate people's diverse needs in toxicity censorship and then build a ChatGPT-based censorship tool named DeMod accordingly. DeMod is equipped with the features of explainable Detection and personalized Modification, providing fine-grained detection results, detailed explanations, and personalized modification suggestions. We also implemented the tool and recruited 35 Weibo users for evaluation. The results suggest DeMod's multiple strengths like the richness of functionality, the accuracy of censorship, and ease of use. Based on the findings, we further propose several insights into the design of content censorship systems.