# Adversarial Generation and Encoding of Nested Texts

## TL;DR

This paper introduces AGENT, a hierarchical language model that encodes, generates, and refines nested long texts like books by learning vector representations at multiple levels and employing adversarial techniques for improved coherence.

## Contribution

The paper presents a novel hierarchical language model, AGENT, with a new adversarial approach for long text generation and methods to enhance coherence through vector representation traversal.

## Key findings

- Effective encoding of nested texts into hierarchical vector representations
- Improved long text generation coherence using adversarial training
- Successful application to book-like documents with hierarchical annotations

## Abstract

In this paper we propose a new language model called AGENT, which stands for Adversarial Generation and Encoding of Nested Texts. AGENT is designed for encoding, generating and refining documents that consist of a long and coherent text, such as an entire book, provided they are hierarchically annotated (nested). i.e. divided into sentences, paragraphs and chapters. The core idea of our system is learning vector representations for each level of the text hierarchy (sentences, paragraphs, etc...), and train each such representation to perform 3 tasks: The task of reconstructing the sequence of vectors from a lower level that was used to create the representation, and generalized versions of the Masked Language Modeling (MLM) and "Next Sentence Prediction" tasks from BERT Devlin et al. [2018]. Additionally we present a new adversarial model for long text generation and suggest a way to improve the coherence of the generated text by traversing its vector representation tree.