DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generation

TL;DR

DYPLOC is a novel framework for long-form opinion text generation that employs dynamic content planning and mixed language models to improve coherence and content diversity in generated texts.

Contribution

The paper introduces DYPLOC, a new generation framework that combines dynamic planning with mixed language models and large pre-trained models for diverse, coherent long-form opinion text generation.

Findings

Significantly outperforms baselines in automatic metrics.

Generations are more coherent and content-rich according to human judges.

Effective on datasets from Reddit and New York Times opinion articles.

Abstract

We study the task of long-form opinion text generation, which faces at least two distinct challenges. First, existing neural generation models fall short of coherence, thus requiring efficient content planning. Second, diverse types of information are needed to guide the generator to cover both subjective and objective content. To this end, we propose DYPLOC, a generation framework that conducts dynamic planning of content while generating the output based on a novel design of mixed language models. To enrich the generation with diverse content, we further propose to use large pre-trained models to predict relevant concepts and to generate claims. We experiment with two challenging tasks on newly collected datasets: (1) argument generation with Reddit ChangeMyView, and (2) writing articles using New York Times' Opinion section. Automatic evaluation shows that our model significantly outperforms competitive comparisons. Human judges further confirm that our generations are more coherent with richer content.