Heating Up Quasi-Monte Carlo Graph Random Features: A Diffusion Kernel Perspective

TL;DR

This paper extends quasi-graph random features to various kernels, especially the Diffusion kernel, demonstrating lower variance estimators on specific graph types and exploring the impact of graph structure on performance.

Contribution

It introduces the use of quasi-graph random features with multiple kernels, notably the Diffusion kernel, and analyzes their variance reduction on different graph models.

Findings

q-GRFs achieve lower variance estimators of the Diffusion kernel on Ladder graphs.

Performance depends on the number of rungs in Ladder graphs.

The Diffusion kernel behaves similarly to the 2-regularized Laplacian in this context.

Abstract

We build upon a recently introduced class of quasi-graph random features (q-GRFs), which have demonstrated the ability to yield lower variance estimators of the 2-regularized Laplacian kernel (Choromanski 2023). Our research investigates whether similar results can be achieved with alternative kernel functions, specifically the Diffusion (or Heat), Mat\'ern, and Inverse Cosine kernels. We find that the Diffusion kernel performs most similarly to the 2-regularized Laplacian, and we further explore graph types that benefit from the previously established antithetic termination procedure. Specifically, we explore Erd\H{o}s-R\'enyi and Barab\'asi-Albert random graph models, Binary Trees, and Ladder graphs, with the goal of identifying combinations of specific kernel and graph type that benefit from antithetic termination. We assert that q-GRFs achieve lower variance estimators of the Diffusion (or Heat) kernel on Ladder graphs. However, the number of rungs on the Ladder graphs impacts the algorithm's performance; further theoretical results supporting our experimentation are forthcoming. This work builds upon some of the earliest Quasi-Monte Carlo methods for kernels defined on combinatorial objects, paving the way for kernel-based learning algorithms and future real-world applications in various domains.