BLISS: Bandit Layer Importance Sampling Strategy for Efficient Training of Graph Neural Networks

TL;DR

BLISS is a dynamic sampling strategy using multi-armed bandits to efficiently select informative nodes in GNNs, reducing computational costs while maintaining or improving accuracy on large graphs.

Contribution

Introduces BLISS, a novel bandit-based importance sampling method that adapts node selection in GNNs, enhancing efficiency and performance over static sampling approaches.

Findings

BLISS maintains or exceeds full-batch training accuracy.

It adapts to different GNN architectures like GCNs and GATs.

Reduces computational costs in large graph training.

Abstract

Graph Neural Networks (GNNs) are powerful tools for learning from graph-structured data, but their application to large graphs is hindered by computational costs. The need to process every neighbor for each node creates memory and computational bottlenecks. To address this, we introduce BLISS, a Bandit Layer Importance Sampling Strategy. It uses multi-armed bandits to dynamically select the most informative nodes at each layer, balancing exploration and exploitation to ensure comprehensive graph coverage. Unlike existing static sampling methods, BLISS adapts to evolving node importance, leading to more informed node selection and improved performance. It demonstrates versatility by integrating with both Graph Convolutional Networks (GCNs) and Graph Attention Networks (GATs), adapting its selection policy to their specific aggregation mechanisms. Experiments show that BLISS maintains or exceeds the accuracy of full-batch training.