OpenFGL: A Comprehensive Benchmark for Federated Graph Learning

TL;DR

OpenFGL introduces a comprehensive benchmark for federated graph learning, including diverse datasets, simulation strategies, and algorithms, to facilitate fair evaluation and advance research in this emerging field.

Contribution

It provides the first unified benchmark for FGL, integrating datasets, simulation strategies, and algorithms to standardize evaluation and foster progress.

Findings

FGL algorithms show varied effectiveness across datasets

Benchmark reveals limitations in current FGL methods

Results highlight the importance of data properties in FGL performance

Abstract

Federated graph learning (FGL) is a promising distributed training paradigm for graph neural networks across multiple local systems without direct data sharing. This approach inherently involves large-scale distributed graph processing, which closely aligns with the challenges and research focuses of graph-based data systems. Despite the proliferation of FGL, the diverse motivations from real-world applications, spanning various research backgrounds and settings, pose a significant challenge to fair evaluation. To fill this gap, we propose OpenFGL, a unified benchmark designed for the primary FGL scenarios: Graph-FL and Subgraph-FL. Specifically, OpenFGL includes 42 graph datasets from 18 application domains, 8 federated data simulation strategies that emphasize different graph properties, and 5 graph-based downstream tasks. Additionally, it offers 18 recently proposed SOTA FGL algorithms through a user-friendly API, enabling a thorough comparison and comprehensive evaluation of their effectiveness, robustness, and efficiency. Our empirical results demonstrate the capabilities of FGL while also highlighting its potential limitations, providing valuable insights for future research in this growing field, particularly in fostering greater interdisciplinary collaboration between FGL and data systems.