MOCHA: A Multi-Task Training Approach for Coherent Text Generation from Cognitive Perspective

TL;DR

This paper introduces MOCHA, a multi-task training approach grounded in cognitive writing theory, which improves neural models' ability to generate more coherent texts across various narrative tasks.

Contribution

It proposes a novel multi-task training strategy that incorporates writing subskills like planning and reviewing, enhancing coherence in text generation.

Findings

Outperforms strong baselines in few-shot and fully-supervised settings.

Generates more coherent texts according to human evaluations.

Achieves better results on story, news, and argument generation tasks.

Abstract

Teaching neural models to generate narrative coherent texts is a critical problem. Recent pre-trained language models have achieved promising results, but there is still a gap between human written texts and machine-generated outputs. In this work, we propose a novel multi-task training strategy for coherent text generation grounded on the cognitive theory of writing, which empowers the model to learn essential subskills needed for writing including planning and reviewing besides end-to-end generation. We extensively evaluate our model on three open-ended generation tasks including story generation, news article writing and argument generation. Experiments show that our model achieves better results on both few-shot and fully-supervised settings than strong baselines, and human evaluations confirm that our model can generate more coherent outputs.