Conditional Poisson Stochastic Beam Search
Clara Meister, Afra Amini, Tim Vieira, Ryan Cotterell

TL;DR
This paper introduces Conditional Poisson stochastic beam search, a novel stochastic decoding method for NLP sequence generation that improves diversity and estimator efficiency over existing approaches.
Contribution
The paper proposes a new stochastic decoding method, CPSBS, that replaces deterministic beam search with conditional Poisson sampling, enhancing diversity and estimator consistency.
Findings
CPSBS produces lower variance estimators than SBS.
CPSBS achieves more efficient sampling in high entropy settings.
CPSBS improves diversity in generated sequences.
Abstract
Beam search is the default decoding strategy for many sequence generation tasks in NLP. The set of approximate K-best items returned by the algorithm is a useful summary of the distribution for many applications; however, the candidates typically exhibit high overlap and may give a highly biased estimate for expectations under our model. These problems can be addressed by instead using stochastic decoding strategies. In this work, we propose a new method for turning beam search into a stochastic process: Conditional Poisson stochastic beam search. Rather than taking the maximizing set at each iteration, we sample K candidates without replacement according to the conditional Poisson sampling design. We view this as a more natural alternative to Kool et. al. 2019's stochastic beam search (SBS). Furthermore, we show how samples generated under the CPSBS design can be used to build…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Topic Modeling · Machine Learning and Algorithms
