TL;DR
This paper introduces CAE-T5, a self-supervised transformer model designed to rephrase toxic comments into more civil language, improving moderation support with fluent and content-preserving outputs.
Contribution
The work presents a novel self-supervised transformer model, CAE-T5, for unpaired text style transfer to civil language, trained on large-scale toxicity data.
Findings
CAE-T5 produces more fluent rephrasings.
It better preserves original content.
Outperforms previous style transfer systems.
Abstract
Platforms that support online commentary, from social networks to news sites, are increasingly leveraging machine learning to assist their moderation efforts. But this process does not typically provide feedback to the author that would help them contribute according to the community guidelines. This is prohibitively time-consuming for human moderators to do, and computational approaches are still nascent. This work focuses on models that can help suggest rephrasings of toxic comments in a more civil manner. Inspired by recent progress in unpaired sequence-to-sequence tasks, a self-supervised learning model is introduced, called CAE-T5. CAE-T5 employs a pre-trained text-to-text transformer, which is fine tuned with a denoising and cyclic auto-encoder loss. Experimenting with the largest toxicity detection dataset to date (Civil Comments) our model generates sentences that are more…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
