Positioning yourself in the maze of Neural Text Generation: A Task-Agnostic Survey

TL;DR

This survey comprehensively reviews task-agnostic modeling techniques in neural text generation, covering learning paradigms, pretraining, and decoding, to help researchers understand the field's landscape and inter-task impacts.

Contribution

It provides an abstraction of fundamental modeling approaches and challenges in neural text generation, serving as a comprehensive guide for researchers to position their work.

Findings

Summarizes key modeling techniques and paradigms.

Identifies major challenges in the field.

Provides a unified perspective across various generation tasks.

Abstract

Neural text generation metamorphosed into several critical natural language applications ranging from text completion to free form narrative generation. In order to progress research in text generation, it is critical to absorb the existing research works and position ourselves in this massively growing field. Specifically, this paper surveys the fundamental components of modeling approaches relaying task agnostic impacts across various generation tasks such as storytelling, summarization, translation etc., In this context, we present an abstraction of the imperative techniques with respect to learning paradigms, pretraining, modeling approaches, decoding and the key challenges outstanding in the field in each of them. Thereby, we deliver a one-stop destination for researchers in the field to facilitate a perspective on where to situate their work and how it impacts other closely related generation tasks.