Graph-to-Sequence Neural Machine Translation

TL;DR

This paper introduces Graph-Transformer, a graph-to-sequence neural machine translation model that explicitly captures subgraph information at various dependency levels, improving translation quality over standard Transformer models.

Contribution

It presents a novel graph-based SAN model for NMT that explicitly models subgraphs of different orders, enhancing the ability to capture dependency structures.

Findings

Improves BLEU scores by 1.1 on WMT14 English-German

Enhances translation quality on IWSLT14 German-English

Effectively captures multi-level dependency information

Abstract

Neural machine translation (NMT) usually works in a seq2seq learning way by viewing either source or target sentence as a linear sequence of words, which can be regarded as a special case of graph, taking words in the sequence as nodes and relationships between words as edges. In the light of the current NMT models more or less capture graph information among the sequence in a latent way, we present a graph-to-sequence model facilitating explicit graph information capturing. In detail, we propose a graph-based SAN-based NMT model called Graph-Transformer by capturing information of subgraphs of different orders in every layers. Subgraphs are put into different groups according to their orders, and every group of subgraphs respectively reflect different levels of dependency between words. For fusing subgraph representations, we empirically explore three methods which weight different groups of subgraphs of different orders. Results of experiments on WMT14 English-German and IWSLT14 German-English show that our method can effectively boost the Transformer with an improvement of 1.1 BLEU points on WMT14 English-German dataset and 1.0 BLEU points on IWSLT14 German-English dataset.