General Graph Random Features
Isaac Reid, Krzysztof Choromanski, Eli Berger, Adrian Weller

TL;DR
This paper introduces a scalable, random walk-based algorithm for estimating graph functions efficiently, enabling large-scale graph kernel learning with theoretical guarantees and diverse applications.
Contribution
The paper presents a novel, scalable algorithm for unbiased graph function estimation using random walks, with neural network modulation for improved accuracy and efficiency.
Findings
Subquadratic time complexity for graph kernel estimation
Neural network modulation improves kernel quality
Effective in large-scale graph learning tasks
Abstract
We propose a novel random walk-based algorithm for unbiased estimation of arbitrary functions of a weighted adjacency matrix, coined universal graph random features (u-GRFs). This includes many of the most popular examples of kernels defined on the nodes of a graph. Our algorithm enjoys subquadratic time complexity with respect to the number of nodes, overcoming the notoriously prohibitive cubic scaling of exact graph kernel evaluation. It can also be trivially distributed across machines, permitting learning on much larger networks. At the heart of the algorithm is a modulation function which upweights or downweights the contribution from different random walks depending on their lengths. We show that by parameterising it with a neural network we can obtain u-GRFs that give higher-quality kernel estimates or perform efficient, scalable kernel learning. We provide robust theoretical…
Peer Reviews
Decision·ICLR 2024 poster
This result is a nice extension of the previous result due to Choromanski (2023) allowing us to compute approximations of a variety of graph kernels.
I do not understand the proof of the main Theorem 2.1 see below. The experiments could be improved. The authors mainly compare the results given by their algorithm to the real kernel, i.e., measure the approximation error due to their algorithm. It would be more convincing to have used their algorithm on a 'real' graph problem with a large number of nodes and where standard kernel methods are not possible.
- This paper extends the recent work of Choromanski (2023) on random walk-based graph random features. Compared to the previous work, this paper has two notable innovations. The first is an introduction of a modulation function which enables estimation of arbitrary functions of the weighted adjacency matrix, the second is an extension to a parametrized setting with a generalization bound. While this work is not the first to study graph random features, it has an easy-to-identify contribution whi
- The experiments are carried over small graphs. It would be nice to include some experiments on larger graphs, e.g. N=10,000. For example, fix the Erdos-Renyi model but vary the number of nodes. I wonder if and how the approximation error would scale with N, while keeping everything else fixed. - For experiments on node clustering, Table 2 shows the difference between u-GRFs and the exact kernel. It would be nice to have another table that shows the actual clustering accuracy when compared aga
- The paper brings two main contributions. First the introduction of the notion of modulation function and its use in a random walk algorithm to estimate arbitrary functions of the weighted adjacency matrix of a graph. Second, the possibility to learn using neural network this modularity function to automatically selects a graph representation well suited for some downstream task. Therefore, this work not only allows to reduce the computational burden to solve some important graph related ques
- The authors highlight several times in the paper that one major issue regarding in terms of computational complexity is to inverse the matrix K(W) (where W is the weighted adjacency matrix of the graph). However, in the numerical experiments presented in the paper, the authors do not propose an example of application of their method to inverse such matrix K. It might be relevant to add such experiment in the paper or at least to explain how the method can be used to efficiently compute the inv
Code & Models
Videos
Taxonomy
TopicsAdvanced Graph Neural Networks · Complex Network Analysis Techniques · Machine Learning and ELM
