GeDi: Generative Discriminator Guided Sequence Generation

TL;DR

GeDi is a method that uses smaller discriminative models to guide large language models for safer, more controllable text generation, achieving faster speeds and zero-shot topic control.

Contribution

Introduces GeDi, a novel approach that guides large language models with smaller discriminators for improved safety and controllability, including zero-shot topic control.

Findings

GeDi outperforms state-of-the-art controllability methods.

Generation speed is more than 30 times faster with GeDi.

Successfully reduces toxicity in GPT-2 without losing linguistic quality.

Abstract

While large-scale language models (LMs) are able to imitate the distribution of natural language well enough to generate realistic text, it is difficult to control which regions of the distribution they generate. This is especially problematic because datasets used for training large LMs usually contain significant toxicity, hate, bias, and negativity. We propose GeDi as an efficient method for using smaller LMs as generative discriminators to guide generation from large LMs to make them safer and more controllable. GeDi guides generation at each step by computing classification probabilities for all possible next tokens via Bayes rule by normalizing over two class-conditional distributions; one conditioned on the desired attribute, or control code, and another conditioned on the undesired attribute, or anti control code. We find that GeDi gives stronger controllability than the state of the art method while also achieving generation speeds more than 30 times faster. Additionally, training GeDi on only four topics allows us to controllably generate new topics zero-shot from just a keyword, unlocking a new capability that previous controllable generation methods do not have. Lastly, we show that GeDi can make GPT-2 (1.5B parameters) significantly less toxic without sacrificing linguistic quality, making it by far the most practical existing method for detoxifying large language models while maintaining a fast generation speed.