Higher-order Sparse Convolutions in Graph Neural Networks

TL;DR

This paper introduces a novel higher-order sparse convolution method for Graph Neural Networks, leveraging Sobolev norms to enhance expressive power while maintaining scalability for large graphs.

Contribution

The paper proposes S-SobGNN, a new higher-order sparse convolution technique based on Sobolev norms, enabling more expressive GNNs for large-scale graph data.

Findings

S-SobGNN achieves competitive results across various semi-supervised learning tasks.

The method effectively captures higher-order relationships without scalability issues.

Performance is comparable or superior to state-of-the-art GNN approaches.

Abstract

Graph Neural Networks (GNNs) have been applied to many problems in computer sciences. Capturing higher-order relationships between nodes is crucial to increase the expressive power of GNNs. However, existing methods to capture these relationships could be infeasible for large-scale graphs. In this work, we introduce a new higher-order sparse convolution based on the Sobolev norm of graph signals. Our Sparse Sobolev GNN (S-SobGNN) computes a cascade of filters on each layer with increasing Hadamard powers to get a more diverse set of functions, and then a linear combination layer weights the embeddings of each filter. We evaluate S-SobGNN in several applications of semi-supervised learning. S-SobGNN shows competitive performance in all applications as compared to several state-of-the-art methods.