Revisiting Embeddings for Graph Neural Networks

TL;DR

This paper investigates how different embedding extraction techniques influence the performance of Graph Neural Networks across various data types, highlighting the dependency of GNN accuracy on embedding style.

Contribution

It provides a systematic analysis of embedding quality and its impact on GNN performance, emphasizing the dominance of BoW embeddings in current datasets.

Findings

GNN performance varies significantly with embedding style.

BoW embeddings are prevalent in graph datasets.

Embedding choice critically affects GNN accuracy.

Abstract

Current graph representation learning techniques use Graph Neural Networks (GNNs) to extract features from dataset embeddings. In this work, we examine the quality of these embeddings and assess how changing them can affect the accuracy of GNNs. We explore different embedding extraction techniques for both images and texts; and find that the performance of different GNN architectures is dependent on the embedding style used. We see a prevalence of bag of words (BoW) embeddings and text classification tasks in available graph datasets. Given the impact embeddings has on GNN performance. this leads to a phenomenon that GNNs being optimised for BoW vectors.