Sketch-Augmented Features Improve Learning Long-Range Dependencies in Graph Neural Networks

TL;DR

This paper introduces Sketched Random Features, a method that injects global, topology-agnostic embeddings into GNNs to better capture long-range dependencies and improve performance on real-world tasks.

Contribution

The authors propose a novel global embedding technique for GNNs that alleviates oversquashing and oversmoothing, enhancing long-range dependency learning.

Findings

Improved GNN performance on real-world tasks.

Global embeddings alleviate oversquashing and oversmoothing.

Method complements existing techniques like positional encodings.

Abstract

Graph Neural Networks learn on graph-structured data by iteratively aggregating local neighborhood information. While this local message passing paradigm imparts a powerful inductive bias and exploits graph sparsity, it also yields three key challenges: (i) oversquashing of long-range information, (ii) oversmoothing of node representations, and (iii) limited expressive power. In this work we inject randomized global embeddings of node features, which we term \textit{Sketched Random Features}, into standard GNNs, enabling them to efficiently capture long-range dependencies. The embeddings are unique, distance-sensitive, and topology-agnostic -- properties which we analytically and empirically show alleviate the aforementioned limitations when injected into GNNs. Experimental results on real-world graph learning tasks confirm that this strategy consistently improves performance over baseline GNNs, offering both a standalone solution and a complementary enhancement to existing techniques such as graph positional encodings. Our source code is available at \href{https://github.com/ryienh/sketched-random-features}{https://github.com/ryienh/sketched-random-features}.