What Do You Get When You Cross Beam Search with Nucleus Sampling?

TL;DR

This paper introduces two deterministic algorithms combining beam search with nucleus sampling for natural language generation, achieving comparable performance to standard beam search on translation and summarization tasks.

Contribution

It proposes p-exact search and dynamic beam search algorithms that integrate nucleus sampling with beam search, offering deterministic alternatives with similar effectiveness.

Findings

Both algorithms match beam search performance.

They work effectively on translation and summarization tasks.

The methods provide deterministic options for probabilistic search techniques.

Abstract

We combine beam search with the probabilistic pruning technique of nucleus sampling to create two deterministic nucleus search algorithms for natural language generation. The first algorithm, p-exact search, locally prunes the next-token distribution and performs an exact search over the remaining space. The second algorithm, dynamic beam search, shrinks and expands the beam size according to the entropy of the candidate's probability distribution. Despite the probabilistic intuition behind nucleus search, experiments on machine translation and summarization benchmarks show that both algorithms reach the same performance levels as standard beam search.