Calibrate and Debias Layer-wise Sampling for Graph Convolutional Networks

TL;DR

This paper improves layer-wise sampling in GCNs by addressing suboptimal probabilities and bias, leading to more accurate and efficient node embedding aggregation, supported by theoretical analysis and experiments.

Contribution

It introduces a new sampling probability principle and a debiasing algorithm for layer-wise GCN sampling methods, enhancing their accuracy and efficiency.

Findings

Reduced estimation variance in sampling.

Improved node embedding accuracy on benchmarks.

Open-source code available for implementation.

Abstract

Multiple sampling-based methods have been developed for approximating and accelerating node embedding aggregation in graph convolutional networks (GCNs) training. Among them, a layer-wise approach recursively performs importance sampling to select neighbors jointly for existing nodes in each layer. This paper revisits the approach from a matrix approximation perspective, and identifies two issues in the existing layer-wise sampling methods: suboptimal sampling probabilities and estimation biases induced by sampling without replacement. To address these issues, we accordingly propose two remedies: a new principle for constructing sampling probabilities and an efficient debiasing algorithm. The improvements are demonstrated by extensive analyses of estimation variance and experiments on common benchmarks. Code and algorithm implementations are publicly available at https://github.com/ychen-stat-ml/GCN-layer-wise-sampling .