Content Augmented Graph Neural Networks

TL;DR

This paper introduces a novel approach to enhance graph neural networks by integrating content-based embeddings at higher layers, improving their ability to utilize node content effectively.

Contribution

The paper proposes augmenting GNN node embeddings with content-derived embeddings at higher layers, addressing the diminishing influence of initial content features.

Findings

Enhanced accuracy on real-world datasets

Effective integration of content embeddings in GNNs

Improved node classification performance

Abstract

In recent years, graph neural networks (GNNs) have become a popular tool for solving various problems over graphs. In these models, the link structure of the graph is typically exploited and nodes' embeddings are iteratively updated based on adjacent nodes. Nodes' contents are used solely in the form of feature vectors, served as nodes' first-layer embeddings. However, the filters or convolutions, applied during iterations/layers to these initial embeddings lead to their impact diminish and contribute insignificantly to the final embeddings. In order to address this issue, in this paper we propose augmenting nodes' embeddings by embeddings generated from their content, at higher GNN layers. More precisely, we propose models wherein a structural embedding using a GNN and a content embedding are computed for each node. These two are combined using a combination layer to form the embedding of a node at a given layer layer. We suggest methods such as using an auto-encoder or building a content graph, to generate content embeddings. In the end, by conducting experiments over several real-world datasets, we demonstrate the high accuracy and performance of our models.