Determinantal Beam Search
Clara Meister, Martina Forster, Ryan Cotterell

TL;DR
Determinantal beam search reformulates traditional beam search as a diverse subset selection process using determinantal point processes, improving the generation of varied and representative sequences in neural language models.
Contribution
It introduces a novel reformulation of beam search leveraging DPPs to explicitly promote diversity in generated sets, addressing high overlap issues.
Findings
Competitive performance in language generation tasks
Enhanced diversity in generated sequences
General approach applicable to various subset selection problems
Abstract
Beam search is a go-to strategy for decoding neural sequence models. The algorithm can naturally be viewed as a subset optimization problem, albeit one where the corresponding set function does not reflect interactions between candidates. Empirically, this leads to sets often exhibiting high overlap, e.g., strings may differ by only a single word. Yet in use-cases that call for multiple solutions, a diverse or representative set is often desired. To address this issue, we propose a reformulation of beam search, which we call determinantal beam search. Determinantal beam search has a natural relationship to determinantal point processes (DPPs), models over sets that inherently encode intra-set interactions. By posing iterations in beam search as a series of subdeterminant maximization problems, we can turn the algorithm into a diverse subset selection process. In a case study, we use the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
