Graph Neural Networks for Natural Language Processing: A Survey

TL;DR

This survey comprehensively reviews the application of Graph Neural Networks in NLP, organizing existing research into a new taxonomy, and discussing challenges and future directions for this emerging field.

Contribution

It introduces a novel taxonomy of GNNs for NLP and provides a comprehensive overview of applications, datasets, and open challenges in the field.

Findings

GNNs are increasingly used in diverse NLP tasks.

A new taxonomy categorizes GNN approaches in NLP.

Identification of key challenges and future research directions.

Abstract

Deep learning has become the dominant approach in coping with various tasks in Natural LanguageProcessing (NLP). Although text inputs are typically represented as a sequence of tokens, there isa rich variety of NLP problems that can be best expressed with a graph structure. As a result, thereis a surge of interests in developing new deep learning techniques on graphs for a large numberof NLP tasks. In this survey, we present a comprehensive overview onGraph Neural Networks(GNNs) for Natural Language Processing. We propose a new taxonomy of GNNs for NLP, whichsystematically organizes existing research of GNNs for NLP along three axes: graph construction,graph representation learning, and graph based encoder-decoder models. We further introducea large number of NLP applications that are exploiting the power of GNNs and summarize thecorresponding benchmark datasets, evaluation metrics, and open-source codes. Finally, we discussvarious outstanding challenges for making the full use of GNNs for NLP as well as future researchdirections. To the best of our knowledge, this is the first comprehensive overview of Graph NeuralNetworks for Natural Language Processing.