Do We Really Need Graph Convolution During Training? Light Post-Training Graph-ODE for Efficient Recommendation

TL;DR

This paper questions the necessity of graph convolutions during training in recommender systems and introduces LightGODE, a post-training graph ODE method that reduces training time while maintaining or improving recommendation quality.

Contribution

The paper proposes LightGODE, a novel post-training graph convolution approach using a graph ODE, which bypasses intensive message passing and addresses embedding discrepancy issues.

Findings

LightGODE reduces training time significantly.

It outperforms GCN-based models in effectiveness.

It mitigates embedding discrepancy in deep graph models.

Abstract

The efficiency and scalability of graph convolution networks (GCNs) in training recommender systems (RecSys) have been persistent concerns, hindering their deployment in real-world applications. This paper presents a critical examination of the necessity of graph convolutions during the training phase and introduces an innovative alternative: the Light Post-Training Graph Ordinary-Differential-Equation (LightGODE). Our investigation reveals that the benefits of GCNs are more pronounced during testing rather than training. Motivated by this, LightGODE utilizes a novel post-training graph convolution method that bypasses the computation-intensive message passing of GCNs and employs a non-parametric continuous graph ordinary-differential-equation (ODE) to dynamically model node representations. This approach drastically reduces training time while achieving fine-grained post-training graph convolution to avoid the distortion of the original training embedding space, termed the embedding discrepancy issue. We validate our model across several real-world datasets of different scales, demonstrating that LightGODE not only outperforms GCN-based models in terms of efficiency and effectiveness but also significantly mitigates the embedding discrepancy commonly associated with deeper graph convolution layers. Our LightGODE challenges the prevailing paradigms in RecSys training and suggests re-evaluating the role of graph convolutions, potentially guiding future developments of efficient large-scale graph-based RecSys.