ContiGuard: A Framework for Continual Toxicity Detection Against Evolving Evasive Perturbations

TL;DR

ContiGuard introduces a continual learning framework for toxicity detection that adapts to evolving evasive perturbations by enhancing semantic understanding and discriminative feature learning, maintaining detection robustness over time.

Contribution

This work is the first to propose a continual learning framework specifically designed for toxicity detection against evolving perturbations, incorporating semantic enrichment and discriminability strategies.

Findings

Improves detection robustness against evolving evasive tactics.

Enhances semantic comprehension of perturbed toxic content.

Maintains high detection accuracy over time despite perturbations.

Abstract

Toxicity detection mitigates the dissemination of toxic content (e.g., hateful comments, posts, and messages within online social actions) to safeguard a healthy online social environment. However, malicious users persistently develop evasive perturbations to disguise toxic content and evade detectors. Traditional detectors or methods are static over time and are inadequate in addressing these evolving evasion tactics. Thus, continual learning emerges as a logical approach to dynamically update detection ability against evolving perturbations. Nevertheless, disparities across perturbations hinder the detector's continual learning on perturbed text. More importantly, perturbation-induced noises distort semantics to degrade comprehension and also impair critical feature learning to render detection sensitive to perturbations. These amplify the challenge of continual learning against evolving perturbations. In this work, we present ContiGuard, the first framework tailored for continual learning of the detector on time-evolving perturbed text (termed continual toxicity detection) to enable the detector to continually update capability and maintain sustained resilience against evolving perturbations. Specifically, to boost the comprehension, we present an LLM-powered semantic enriching strategy, where we dynamically incorporate possible meaning and toxicity-related clues excavated by LLM into the perturbed text to improve the comprehension. To mitigate non-critical features and amplify critical ones, we propose a discriminability-driven feature learning strategy, where we strengthen discriminative features while suppressing the less-discriminative ones to shape a robust classification boundary for detection...