Leashing the Inner Demons: Self-Detoxification for Language Models

TL;DR

This paper investigates the causes of toxicity in language models and introduces a simple self-detoxification method that reduces harmful outputs without external data, maintaining quality.

Contribution

It presents a novel self-detoxification technique enabling language models to reduce toxicity independently, without requiring additional training data or external discriminators.

Findings

The method effectively reduces toxicity across various prompts and decoding strategies.

Self-detoxification maintains high-quality language generation.

Compared to supervised approaches, it achieves better toxicity mitigation.

Abstract

Language models (LMs) can reproduce (or amplify) toxic language seen during training, which poses a risk to their practical application. In this paper, we conduct extensive experiments to study this phenomenon. We analyze the impact of prompts, decoding strategies and training corpora on the output toxicity. Based on our findings, we propose a simple yet effective method for language models to "detoxify" themselves without an additional large corpus or external discriminator. Compared to a supervised baseline, our proposed method shows better toxicity reduction with good generation quality in the generated content under multiple settings. Warning: some examples shown in the paper may contain uncensored offensive content.