BSDAR: Beam Search Decoding with Attention Reward in Neural Keyphrase Generation

TL;DR

This paper proposes a novel beam search decoding method with an attention-based reward to improve the diversity and length of generated keyphrases in neural models, addressing common decoding issues.

Contribution

It introduces a reward-based beam search strategy that enhances diversity and length control in neural keyphrase generation, outperforming traditional methods.

Findings

Improved keyphrase generation accuracy for present and absent keyphrases.

Enhanced diversity and length control in generated sequences.

Significant performance gains over baseline decoding methods.

Abstract

This study mainly investigates two common decoding problems in neural keyphrase generation: sequence length bias and beam diversity. To tackle the problems, we introduce a beam search decoding strategy based on word-level and ngram-level reward function to constrain and refine Seq2Seq inference at test time. Results show that our simple proposal can overcome the algorithm bias to shorter and nearly identical sequences, resulting in a significant improvement of the decoding performance on generating keyphrases that are present and absent in source text.