Massive-scale Decoding for Text Generation using Lattices

TL;DR

This paper introduces a novel lattice-based search algorithm for neural text generation that efficiently encodes thousands of diverse, high-quality options, surpassing traditional beam search in diversity and efficiency.

Contribution

It presents a best-first search algorithm with hypothesis recombination to generate massive, diverse text options in a single lattice structure, improving over existing methods.

Findings

Encodes thousands of diverse options in a single lattice

Improves efficiency over beam search

Maintains grammaticality and quality of generated options

Abstract

Conditional neural text generation models generate high-quality outputs, but often concentrate around a mode when what we really want is a diverse set of options. We present a search algorithm to construct lattices encoding a massive number of generation options. First, we restructure decoding as a best-first search, which explores the space differently than beam search and improves efficiency by avoiding pruning paths. Second, we revisit the idea of hypothesis recombination: we can identify pairs of similar generation candidates during search and merge them as an approximation. On both summarization and machine translation, we show that our algorithm encodes thousands of diverse options that remain grammatical and high-quality into one lattice. This algorithm provides a foundation for building downstream generation applications on top of massive-scale diverse outputs.