Naver Labs Europe's Systems for the Document-Level Generation and Translation Task at WNGT 2019

TL;DR

This paper presents a transfer learning approach leveraging large-scale document-level data to improve neural models for long text generation and document-level translation, achieving state-of-the-art results at WNGT 2019.

Contribution

It introduces a transfer learning method that fine-tunes document-based MT models for NLG and MT with metadata, outperforming previous methods without data selection or planning.

Findings

Outperforms previous state-of-the-art on Rotowire NLG task

Ranks first in all tracks at WNGT 2019

Demonstrates effectiveness of transfer learning across tasks

Abstract

Recently, neural models led to significant improvements in both machine translation (MT) and natural language generation tasks (NLG). However, generation of long descriptive summaries conditioned on structured data remains an open challenge. Likewise, MT that goes beyond sentence-level context is still an open issue (e.g., document-level MT or MT with metadata). To address these challenges, we propose to leverage data from both tasks and do transfer learning between MT, NLG, and MT with source-side metadata (MT+NLG). First, we train document-based MT systems with large amounts of parallel data. Then, we adapt these models to pure NLG and MT+NLG tasks by fine-tuning with smaller amounts of domain-specific data. This end-to-end NLG approach, without data selection and planning, outperforms the previous state of the art on the Rotowire NLG task. We participated to the "Document Generation and Translation" task at WNGT 2019, and ranked first in all tracks.