PromptGCN: Bridging Subgraph Gaps in Lightweight GCNs

TL;DR

PromptGCN introduces a prompt-based approach to enhance lightweight GCNs by bridging subgraph gaps, significantly improving accuracy on large-scale graphs while maintaining low memory usage.

Contribution

The paper proposes PromptGCN, a novel prompt-based method that incorporates global information into subgraph training, improving accuracy of lightweight GCNs.

Findings

PromptGCN outperforms baseline methods on seven large-scale graphs.

It improves accuracy by up to 5.48% on the Flickr dataset.

PromptGCN can be combined with any subgraph sampling method for better performance.

Abstract

Graph Convolutional Networks (GCNs) are widely used in graph-based applications, such as social networks and recommendation systems. Nevertheless, large-scale graphs or deep aggregation layers in full-batch GCNs consume significant GPU memory, causing out of memory (OOM) errors on mainstream GPUs (e.g., 29GB memory consumption on the Ogbnproducts graph with 5 layers). The subgraph sampling methods reduce memory consumption to achieve lightweight GCNs by partitioning the graph into multiple subgraphs and sequentially training GCNs on each subgraph. However, these methods yield gaps among subgraphs, i.e., GCNs can only be trained based on subgraphs instead of global graph information, which reduces the accuracy of GCNs. In this paper, we propose PromptGCN, a novel prompt-based lightweight GCN model to bridge the gaps among subgraphs. First, the learnable prompt embeddings are designed to obtain global information. Then, the prompts are attached into each subgraph to transfer the global information among subgraphs. Extensive experimental results on seven largescale graphs demonstrate that PromptGCN exhibits superior performance compared to baselines. Notably, PromptGCN improves the accuracy of subgraph sampling methods by up to 5.48% on the Flickr dataset. Overall, PromptGCN can be easily combined with any subgraph sampling method to obtain a lightweight GCN model with higher accuracy.