ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information

TL;DR

ToxiCraft is a new framework that generates realistic synthetic harmful content to improve NLP models' ability to detect toxic information, especially in low-resource settings, by addressing data scarcity and definition inconsistencies.

Contribution

It introduces a novel data synthesis framework that creates diverse, realistic toxic examples from limited seed data, enhancing model robustness and adaptability.

Findings

Improved detection model robustness and accuracy.

Generated diverse synthetic toxic data effectively.

Surpassed or matched gold-standard labels in experiments.

Abstract

In different NLP tasks, detecting harmful content is crucial for online environments, especially with the growing influence of social media. However, previous research has two main issues: 1) a lack of data in low-resource settings, and 2) inconsistent definitions and criteria for judging harmful content, requiring classification models to be robust to spurious features and diverse. We propose Toxicraft, a novel framework for synthesizing datasets of harmful information to address these weaknesses. With only a small amount of seed data, our framework can generate a wide variety of synthetic, yet remarkably realistic, examples of toxic information. Experimentation across various datasets showcases a notable enhancement in detection model robustness and adaptability, surpassing or close to the gold labels.