IFDID: Information Filter upon Diversity-Improved Decoding for Diversity-Faithfulness Tradeoff in NLG
Han Meng, Xiaosong He, Zexing Chen, Feng Zhou

TL;DR
The paper introduces IFDID, a novel decoding strategy for natural language generation that balances diversity and faithfulness by filtering tokens based on their information content, outperforming existing methods.
Contribution
It proposes a two-stage Enhance-Filter decoding framework that improves diversity and faithfulness tradeoff in NLG tasks, achieving state-of-the-art results.
Findings
Achieves 1.24 higher ROUGE score for faithfulness.
Demonstrates 62.5% higher diversity on Dist-2.
Outperforms baseline methods on multiple benchmarks.
Abstract
Some Natural Language Generation (NLG) tasks require both faithfulness and diversity. The decoding strategy is intensively related to the quality of the generated text. Strategies such as beam search, greedy search, etc., perform with low diversity and high repetition. On the other hand, guided decoding, the solution towards diversity, may generate unfaithful expressions. To this end, this paper presents Information Filter upon Diversity-Improved Decoding (IFDID) to obtain the tradeoff between diversity and faithfulness. IFDID is a two-stage decoding strategy leveraging the proposed Enhance-Filter framework, which achieves the tradeoff by increasing the probabilities of some typical tokens being selected and subsequently filtering them by their information amount. To verify the effectiveness, we compare our method with other baselines on related CommonGEN, RocStories and AdGen…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Topic Modeling · Multimodal Machine Learning Applications
