TransSent: Towards Generation of Structured Sentences with Discourse Marker

TL;DR

TransSent introduces a novel approach to generate structured sentences by explicitly modeling discourse relationships as translation tasks, improving the quality and scalability of structured text generation.

Contribution

The paper proposes TransSent, a model that explicitly separates semantic and structural modeling, and interprets discourse relationships as translation in embedding space for structured sentence generation.

Findings

TransSent effectively generates high-quality structured sentences.

The model demonstrates scalability across different text generation tasks.

Automatic and human evaluations confirm the approach's effectiveness.

Abstract

Structured sentences are important expressions in human writings and dialogues. Previous works on neural text generation fused semantic and structural information by encoding the entire sentence into a mixed hidden representation. However, when a generated sentence becomes complicated, the structure is difficult to be properly maintained. To alleviate this problem, we explicitly separate the modeling process of semantic and structural information. Intuitively, humans generate structured sentences by directly connecting discourses with discourse markers (such as and, but, etc.). Therefore, we propose a task that mimics this process, called discourse transfer. This task represents a structured sentence as (head discourse, discourse marker, tail discourse), and aims at tail discourse generation based on head discourse and discourse marker. We also propose a corresponding model called TransSent, which interprets the relationship between two discourses as a translation1 from the head discourse to the tail discourse in the embedding space. We experiment TransSent not only in discourse transfer task but also in free text generation and dialogue generation tasks. Automatic and human evaluation results show that TransSent can generate structured sentences with high quality, and has certain scalability in different tasks.