Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension
Chao Zhao, Wenlin Yao, Dian Yu, Kaiqiang Song, Dong Yu, Jianshu Chen

TL;DR
This paper introduces a novel narrative-guided pre-training strategy for zero-shot dialogue comprehension, utilizing a newly constructed dialogue-narrative corpus to enhance understanding of dialogues through diverse capabilities.
Contribution
The paper proposes a new narrative pre-training approach for dialogue comprehension and creates a dialogue-narrative corpus by aligning movie subtitles with synopses.
Findings
Achieves superior zero-shot performance on dialogue tasks
Demonstrates stronger fine-grained dialogue comprehension
Validates effectiveness of narrative pre-training strategy
Abstract
Comprehending a dialogue requires a model to capture diverse kinds of key information in the utterances, which are either scattered around or implicitly implied in different turns of conversations. Therefore, dialogue comprehension requires diverse capabilities such as paraphrasing, summarizing, and commonsense reasoning. Towards the objective of pre-training a zero-shot dialogue comprehension model, we develop a novel narrative-guided pre-training strategy that learns by narrating the key information from a dialogue input. However, the dialogue-narrative parallel corpus for such a pre-training strategy is currently unavailable. For this reason, we first construct a dialogue-narrative parallel corpus by automatically aligning movie subtitles and their synopses. We then pre-train a BART model on the data and evaluate its performance on four dialogue-based tasks that require…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Topic Modeling · Natural Language Processing Techniques
MethodsMulti-Head Attention · Linear Layer · Dense Connections · Refunds@Expedia|||How do I get a full refund from Expedia? · Dropout · Layer Normalization · Adam · Byte Pair Encoding · Softmax · Attention Is All You Need
