GraphViz2Vec: A Structure-aware Feature Generation Model to Improve Classification in GNNs

TL;DR

GraphViz2Vec introduces a novel structure-aware feature generation method that enhances GNN initial embeddings, enabling better classification performance with fewer layers and reducing over-smoothing issues.

Contribution

The paper presents a new feature extraction approach using energy diagrams and image classification to improve GNN initial embeddings and classification accuracy.

Findings

Achieves a mean increase of 4.65% in node classification accuracy.

Achieves a mean increase of 2.58% in link classification accuracy.

Some models reach state-of-the-art results with fewer layers.

Abstract

GNNs are widely used to solve various tasks including node classification and link prediction. Most of the GNN architectures assume the initial embedding to be random or generated from popular distributions. These initial embeddings require multiple layers of transformation to converge into a meaningful latent representation. While number of layers allow accumulation of larger neighbourhood of a node it also introduce the problem of over-smoothing. In addition, GNNs are inept at representing structural information. For example, the output embedding of a node does not capture its triangles participation. In this paper, we presented a novel feature extraction methodology GraphViz2Vec that can capture the structural information of a node's local neighbourhood to create meaningful initial embeddings for a GNN model. These initial embeddings helps existing models achieve state-of-the-art results in various classification tasks. Further, these initial embeddings help the model to produce desired results with only two layers which in turn reduce the problem of over-smoothing. The initial encoding of a node is obtained from an image classification model trained on multiple energy diagrams of its local neighbourhood. These energy diagrams are generated with the induced sub-graph of the nodes traversed by multiple random walks. The generated encodings increase the performance of existing models on classification tasks (with a mean increase of $4.65\%$ and $2.58\%$ for the node and link classification tasks, respectively), with some models achieving state-of-the-art results.