Boosting Graph Neural Network Expressivity with Learnable Lanczos Constraints

TL;DR

This paper introduces a novel learnable Lanczos-based method to significantly enhance the expressivity of GNNs, enabling better graph distinction and link prediction with less training data and improved efficiency.

Contribution

We propose LLwLC, a lightweight, learnable Lanczos algorithm with linear constraints that improves GNN expressivity by embedding subgraphs into the Laplacian eigenbasis, surpassing traditional message-passing limits.

Findings

Distinguishes graphs beyond 2-WL capabilities.

Achieves 20x and 10x speedups on benchmark datasets.

Requires only 5-10% of training data for comparable performance.

Abstract

Graph Neural Networks (GNNs) excel in handling graph-structured data but often underperform in link prediction tasks compared to classical methods, mainly due to the limitations of the commonly used message-passing principle. Notably, their ability to distinguish non-isomorphic graphs is limited by the 1-dimensional Weisfeiler-Lehman test. Our study presents a novel method to enhance the expressivity of GNNs by embedding induced subgraphs into the graph Laplacian matrix's eigenbasis. We introduce a Learnable Lanczos algorithm with Linear Constraints (LLwLC), proposing two novel subgraph extraction strategies: encoding vertex-deleted subgraphs and applying Neumann eigenvalue constraints. For the former, we demonstrate the ability to distinguish graphs that are indistinguishable by 2-WL, while maintaining efficient time complexity. The latter focuses on link representations enabling differentiation between $k$-regular graphs and node automorphism, a vital aspect for link prediction tasks. Our approach results in an extremely lightweight architecture, reducing the need for extensive training datasets. Empirically, our method improves performance in challenging link prediction tasks across benchmark datasets, establishing its practical utility and supporting our theoretical findings. Notably, LLwLC achieves 20x and 10x speedup by only requiring 5% and 10% data from the PubMed and OGBL-Vessel datasets while comparing to the state-of-the-art.