Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks

TL;DR

This paper introduces a novel graph-to-sequence neural model that effectively converts graph-structured data into sequences, achieving state-of-the-art results across multiple tasks by incorporating edge direction and attention mechanisms.

Contribution

The paper presents a new end-to-end graph-to-sequence model with an improved graph neural network and attention-based decoding, outperforming existing methods.

Findings

Achieves state-of-the-art performance on multiple tasks

Rapid convergence with bi-directional node embedding aggregation

Significantly outperforms existing graph neural networks and Seq2Seq models

Abstract

The celebrated Sequence to Sequence learning (Seq2Seq) technique and its numerous variants achieve excellent performance on many tasks. However, many machine learning tasks have inputs naturally represented as graphs; existing Seq2Seq models face a significant challenge in achieving accurate conversion from graph form to the appropriate sequence. To address this challenge, we introduce a novel general end-to-end graph-to-sequence neural encoder-decoder model that maps an input graph to a sequence of vectors and uses an attention-based LSTM method to decode the target sequence from these vectors. Our method first generates the node and graph embeddings using an improved graph-based neural network with a novel aggregation strategy to incorporate edge direction information in the node embeddings. We further introduce an attention mechanism that aligns node embeddings and the decoding sequence to better cope with large graphs. Experimental results on bAbI, Shortest Path, and Natural Language Generation tasks demonstrate that our model achieves state-of-the-art performance and significantly outperforms existing graph neural networks, Seq2Seq, and Tree2Seq models; using the proposed bi-directional node embedding aggregation strategy, the model can converge rapidly to the optimal performance.