Residual Hyperbolic Graph Convolution Networks

TL;DR

This paper introduces residual hyperbolic graph convolutional networks (R-HGCNs) that incorporate hyperbolic residual connections, product manifolds, and HyperDrop to mitigate over-smoothing and enhance hierarchical graph representation capabilities.

Contribution

The paper proposes a novel R-HGCN architecture with hyperbolic residuals, product manifolds, and HyperDrop, addressing over-smoothing and improving hierarchical graph modeling.

Findings

R-HGCNs effectively prevent over-smoothing in deep layers.

Product manifolds enhance feature extraction from diverse perspectives.

HyperDrop alleviates overfitting without disrupting hyperbolic geometry.

Abstract

Hyperbolic graph convolutional networks (HGCNs) have demonstrated representational capabilities of modeling hierarchical-structured graphs. However, as in general GCNs, over-smoothing may occur as the number of model layers increases, limiting the representation capabilities of most current HGCN models. In this paper, we propose residual hyperbolic graph convolutional networks (R-HGCNs) to address the over-smoothing problem. We introduce a hyperbolic residual connection function to overcome the over-smoothing problem, and also theoretically prove the effectiveness of the hyperbolic residual function. Moreover, we use product manifolds and HyperDrop to facilitate the R-HGCNs. The distinctive features of the R-HGCNs are as follows: (1) The hyperbolic residual connection preserves the initial node information in each layer and adds a hyperbolic identity mapping to prevent node features from being indistinguishable. (2) Product manifolds in R-HGCNs have been set up with different origin points in different components to facilitate the extraction of feature information from a wider range of perspectives, which enhances the representing capability of R-HGCNs. (3) HyperDrop adds multiplicative Gaussian noise into hyperbolic representations, such that perturbations can be added to alleviate the over-fitting problem without deconstructing the hyperbolic geometry. Experiment results demonstrate the effectiveness of R-HGCNs under various graph convolution layers and different structures of product manifolds.