Serving Graph Neural Networks With Distributed Fog Servers For Smart IoT Services

TL;DR

Fograph is a distributed fog computing framework that enhances real-time GNN inference for IoT applications by reducing communication overhead and leveraging nearby fog nodes, outperforming traditional cloud-based methods.

Contribution

The paper introduces Fograph, a novel fog-based GNN serving framework with heterogeneity-aware planning and compression techniques for improved performance.

Findings

Fograph achieves up to 5.39x speedup over cloud serving.

Fograph improves throughput by up to 6.84x.

Prototype evaluation confirms significant performance gains.

Abstract

Graph Neural Networks (GNNs) have gained growing interest in miscellaneous applications owing to their outstanding ability in extracting latent representation on graph structures. To render GNN-based service for IoT-driven smart applications, traditional model serving paradigms usually resort to the cloud by fully uploading geo-distributed input data to remote datacenters. However, our empirical measurements reveal the significant communication overhead of such cloud-based serving and highlight the profound potential in applying the emerging fog computing. To maximize the architectural benefits brought by fog computing, in this paper, we present Fograph, a novel distributed real-time GNN inference framework that leverages diverse and dynamic resources of multiple fog nodes in proximity to IoT data sources. By introducing heterogeneity-aware execution planning and GNN-specific compression techniques, Fograph tailors its design to well accommodate the unique characteristics of GNN serving in fog environments. Prototype-based evaluation and case study demonstrate that Fograph significantly outperforms the state-of-the-art cloud serving and fog deployment by up to 5.39x execution speedup and 6.84x throughput improvement.