SSFG: Stochastically Scaling Features and Gradients for Regularizing Graph Convolutional Networks

TL;DR

This paper introduces SSFG, a stochastic regularization technique that scales features and gradients during training to mitigate oversmoothing in deep graph convolutional networks, leading to improved performance.

Contribution

The paper proposes a novel stochastic regularization method, SSFG, that alleviates oversmoothing in GCNs by scaling features and gradients without increasing parameters.

Findings

SSFG effectively reduces oversmoothing in deep GCNs.

SSFG improves performance across multiple graph tasks and datasets.

Applying stochastic scaling at feature and gradient levels enhances model robustness.

Abstract

Graph convolutional networks have been successfully applied in various graph-based tasks. In a typical graph convolutional layer, node features are updated by aggregating neighborhood information. Repeatedly applying graph convolutions can cause the oversmoothing issue, i.e., node features at deep layers converge to similar values. Previous studies have suggested that oversmoothing is one of the major issues that restrict the performance of graph convolutional networks. In this paper, we propose a stochastic regularization method to tackle the oversmoothing problem. In the proposed method, we stochastically scale features and gradients (SSFG) by a factor sampled from a probability distribution in the training procedure. By explicitly applying a scaling factor to break feature convergence, the oversmoothing issue is alleviated. We show that applying stochastic scaling at the gradient level is complementary to that applied at the feature level to improve the overall performance. Our method does not increase the number of trainable parameters. When used together with ReLU, our SSFG can be seen as a stochastic ReLU activation function. We experimentally validate our SSFG regularization method on three commonly used types of graph networks. Extensive experimental results on seven benchmark datasets for four graph-based tasks demonstrate that our SSFG regularization is effective in improving the overall performance of the baseline graph networks.