DAFOS: Dynamic Adaptive Fanout Optimization Sampler

TL;DR

DAFOS is a dynamic sampling method for GNNs that adaptively adjusts neighbor fanout and prioritizes important nodes, significantly improving training speed and accuracy on benchmark datasets.

Contribution

It introduces a novel adaptive fanout optimization sampler that dynamically adjusts sampling based on model performance and node importance, enhancing GNN scalability.

Findings

3.57x speedup on ogbn-arxiv

12.6x speedup on Reddit

Improved F1 scores on benchmarks

Abstract

Graph Neural Networks (GNNs) are becoming an essential tool for learning from graph-structured data, however uniform neighbor sampling and static fanout settings frequently limit GNNs' scalability and efficiency. In this paper, we propose the Dynamic Adaptive Fanout Optimization Sampler (DAFOS), a novel approach that dynamically adjusts the fanout based on model performance and prioritizes important nodes during training. Our approach leverages node scoring based on node degree to focus computational resources on structurally important nodes, incrementing the fanout as the model training progresses. DAFOS also integrates an early stopping mechanism to halt training when performance gains diminish. Experiments conducted on three benchmark datasets, ogbnarxiv, Reddit, and ogbn-products, demonstrate that our approach significantly improves training speed and accuracy compared to a state-of-the-art approach. DAFOS achieves a 3.57x speedup on the ogbn-arxiv dataset and a 12.6x speedup on the Reddit dataset while improving the F1 score from 68.5% to 71.21% on ogbn-arxiv and from 73.78% to 76.88% on the ogbn-products dataset, respectively. These results highlight the potential of DAFOS as an efficient and scalable solution for large-scale GNN training.