Sentence-Permuted Paragraph Generation
Wenhao Yu, Chenguang Zhu, Tong Zhao, Zhichun Guo, Meng Jiang

TL;DR
This paper introduces PermGen, a novel paragraph generation framework that enhances content diversity by permuting sentence orders and optimizing across all possible permutations, leading to more diverse and higher-quality paragraphs.
Contribution
PermGen is the first model to maximize paragraph likelihood over all sentence permutations, improving diversity and quality in multi-sentence paragraph generation.
Findings
PermGen produces more diverse paragraphs than existing models.
PermGen achieves higher quality in generated content.
Experiments on three benchmarks validate PermGen's effectiveness.
Abstract
Generating paragraphs of diverse contents is important in many applications. Existing generation models produce similar contents from homogenized contexts due to the fixed left-to-right sentence order. Our idea is permuting the sentence orders to improve the content diversity of multi-sentence paragraph. We propose a novel framework PermGen whose objective is to maximize the expected log-likelihood of output paragraph distributions with respect to all possible sentence orders. PermGen uses hierarchical positional embedding and designs new procedures for training, decoding, and candidate ranking in the sentence-permuted generation. Experiments on three paragraph generation benchmarks demonstrate PermGen generates more diverse outputs with a higher quality than existing models.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Multimodal Machine Learning Applications
