Neural Graph Machines: Learning Neural Networks Using Graphs

TL;DR

This paper introduces Neural Graph Machines, a framework that combines neural networks with graph-based label propagation to leverage both labeled and unlabeled data across various neural architectures and graph types.

Contribution

It generalizes graph-augmented neural network training, enabling scalable, efficient learning on large graphs with multiple neural models and graph inputs.

Findings

Outperforms existing methods on multi-label social graph classification

Effective across different neural architectures and graph types

Scalable with linear runtime in the number of edges

Abstract

Label propagation is a powerful and flexible semi-supervised learning technique on graphs. Neural networks, on the other hand, have proven track records in many supervised learning tasks. In this work, we propose a training framework with a graph-regularised objective, namely "Neural Graph Machines", that can combine the power of neural networks and label propagation. This work generalises previous literature on graph-augmented training of neural networks, enabling it to be applied to multiple neural architectures (Feed-forward NNs, CNNs and LSTM RNNs) and a wide range of graphs. The new objective allows the neural networks to harness both labeled and unlabeled data by: (a) allowing the network to train using labeled data as in the supervised setting, (b) biasing the network to learn similar hidden representations for neighboring nodes on a graph, in the same vein as label propagation. Such architectures with the proposed objective can be trained efficiently using stochastic gradient descent and scaled to large graphs, with a runtime that is linear in the number of edges. The proposed joint training approach convincingly outperforms many existing methods on a wide range of tasks (multi-label classification on social graphs, news categorization, document classification and semantic intent classification), with multiple forms of graph inputs (including graphs with and without node-level features) and using different types of neural networks.