A Frustratingly Simple Decoding Method for Neural Text Generation

TL;DR

This paper presents Frustratingly Simple Decoding (FSD), a highly efficient and effective method for neural text generation that penalizes repeated text using an anti-language model, outperforming existing decoding strategies.

Contribution

The paper introduces FSD, a novel decoding method that requires no extra parameters and is computationally efficient, significantly improving text generation quality.

Findings

FSD outperforms nucleus sampling and strong baselines.

FSD is as fast as greedy search.

FSD effectively reduces repetitive text in generated outputs.

Abstract

We introduce a frustratingly simple, super efficient and surprisingly effective decoding method, which we call Frustratingly Simple Decoding (FSD), for neural text generation. The idea behind FSD is straightforward: we build an anti-LM based on previously generated text and use this anti-LM to penalize future generation of what has been generated. The anti-LM can be implemented as simple as an n-gram language model or a vectorized variant. In this way, FSD introduces no extra model parameters and negligible computational overhead (FSD can be as fast as greedy search). Despite the simplicity, FSD is surprisingly effective; Experiments show that FSD can outperform the canonical methods to date (i.e., nucleus sampling) as well as several strong baselines that were proposed recently.